In this post we read chapter 6, about
Temmporal Difference (TD) learning:
If one had to identify one idea as central and novel to reinforcement learning, it would undoubtedly be TD learning... The relationship between TD, DP, and MC methods is a recurring theme in RL.
TD learning is a combination of MC and DP.
- Like MC, TD methods can learn directly from raw experience without a model of the environment’s dynamics.
- Like DP, TD methods update estimates based in part on other learned estimates, without waiting for a final outcome (they bootstrap).
- Bootstrapping: using a guess to make a better guess.
- The master of RL must master TD, MC, DP as one and use them in many combinations.
