POMDP

Deep Q networks have proven to be an easy to implement method for solving control problems in both continuous or large discrete state spaces. The action-specific deep recurrent Q network (ADRQN) introduces an intermediate LSTM layer for remembering action-observation pairs when dealing with partial observability. This post explores a compact PyTorch implementation of the ADRQN including small scale experiments on classical control tasks.

Particle Filters are a useful technique and can be found in many newly published papers about (model-based) POMDP algorithms. In this post, we explore the mathematical framework of filtering and discuss its implications for POMDPs with a simple example.