MDP

In this blog, we will prove the following theorem: Optimal Policy Existence Theorem: For any Markov Decision Process, There exists an optimal policy ${\pi }_{\star }$ that is better than or equal to all other policies, ${\pi }_{\star }\ge \pi ,\mathrm{\forall }\pi$ All optimal policies achieve the optimal value function, $V_{{\pi }_{\star }}(s)=V_{\star }(s)$ All optimal policies achieve the optimal action-value function $Q_{{\pi }_{\star }}(s,a)=Q_{\star }(s,a)$ Definition To simplify the exposition, we first define some basic concepts. ...