Posts

Here are some immature ideas that I come up with when I am taking a walk or having a meal. Feel free to get some inspiration from this post and write some papers. I am glad to see study on these topics. This post will be updated from time to time. The research field underestimates the importance of optimization. You cannot expect one to have perfect memory. What we do every day is just adjusting our cognition. That’s what Mass Editing Memory in a Transformer does! This method remodels knowledge, namely the relation between object. But it does not involve behavioral remodeling, for example the procedure to solve a tedious math problem. Information of relations between texts and images is in cross-attention layers(e.g. prompt to prompt), while the knowledge-related relations are stored in MLP layers(e.g. https://rome.baulab.info/). Are procedural relations stored in cross-attention layers? ...

In this blog, we will prove the following theorem: Optimal Policy Existence Theorem: For any Markov Decision Process, There exists an optimal policy $\pi_\star $ that is better than or equal to all other policies, $ \pi_\star \geq \pi , \forall \pi$ All optimal policies achieve the optimal value function, $V_{\pi_\star}(s)=V_\star(s)$ All optimal policies achieve the optimal action-value function $Q_{\pi_\star}(s,a)=Q_\star(s,a)$ Definition To simplify the exposition, we first define some basic concepts. ...