Let's read: Sutton's RL, week 3, 4, chap 4

We'll first finish chapter 4.1 from week 3, and for week 4, we'll do:

• Sutton & Barto, rest of Chapter 4
• Sutton & Barto, Chapter 5

In this post we do chapter 4.

Dynamic Programming (DP)

Define: DP: a collection of algorithms to compute optimal policies for an MDP model of the environment. 

Classical DP algorithms are of limited utility in reinforcement learning both because of their assumption of a perfect model and because of their great computational expense, but they are still important theoretically. DP provides an essential foundation for the understanding of the methods presented in the rest of this book. In fact, all of these methods can be viewed as attempts to achieve much the same effect as DP, only with less computation and without assuming a perfect model of the environment.

Basically, can't solve Bellman, can't do classical DP, put them up as unachievable ideals and opt for more practical algorithms that can approximate them.