I’ve taken a recent interest in reinforcement learning, particularly regarding Bayesian approaches. There is a taught seminar course at Harvard right now by Finale Doshi-Velez, a recently hired professor who was a postdoc in Ryan Adam’s Harvard Intelligent Probabilistic Systems lab and who works on partially observable MDPs. For anyone curious about how the course fares, I’m making my ongoing notes available online:

I quite enjoy it so far. As with any seminar course, the structure mostly comprises of reading and discussing seminal papers, and which has led to very interesting discussions. The content itself is quite intriguing as authors have integrated concepts from a variety of fields such as game theory.

I’ve found the experiments and practical applications a bit lackluster however. That is, papers have focused on indicating the effiency of their algorithms through toy problems, which isn’t as appealing as, say, the standard for supervised learning. That is, supervised learning typically benchmarks on a more realistic problem such as classifying categories for text documents or handwritten digits. It will be interesting to see how experimental practices develop as reinforcement learning sees more application in realistic domains.

On another note, I also happen to be taking a course at MIT on cognitive robotics (6.834J) by Brian Williams, which has an interesting overlap in terms of where RL algorithms can be applied. There is an older version of the course archived in the MIT OpenCourseWare—no recorded lectures unfortunately. However, there are incredibly useful lecture slides given by the instructor. If given permission, I will release them too.