CS4246 Module Review – AI Planning and Decision Making (16/17 Sem 1)

Overview

• One of the hardest and most theoretical module I have taken in NUS
• The content is a mixture of classical concepts and cutting edge research topics
• Project is also very theoretical, involving more conceptual understanding and reading of research papers rather than implementation.
• Midterm is very early, before recess week. Final exam is even earlier, before exam week. That is why the CORS says the module has no exam, when there is in fact a final exam.

What’s Covered in CS4246

First, the module lay some foundation work by introducing basic decision making concepts such as utility, preference, utility theory and lottery. Those are relatively straight-forward concepts with very minimal formulas to remember.

MDP & POMDP

Then it moves on to the classical topics in the area of robotics and planning, MDP and POMDP. They are one of the major components in this module. This is the part where a lot of conceptual understanding and tedious calculations comes in (the latter being the signature style of Prof. Brian Low, which you would know if you have taken his CS3243 Introduction to AI).

Gaussian Process

But in the middle of MDP and POMDP is the topic of Gaussian process (GP) which really came out of nowhere. It is a statistical model used in machine learning and it is actively being researched in NUS. If you have taken or is taking CS3244 Machine Learning, then you should be able to make some connections using the concept of kernel (or covariance matrix). But being a cutting edge research topic, it is still pretty hard to understand. Luckily, for our cohort, GP only appeared in the project, but not exams.

Reinforcement Learning, Active Learning & Bayesian Optimization

In the second half of the semester, the module briefly introduces a few classical Reinforcement Learning techniques, including Adaptive Dynamic Programing (ADP), Temporal-Difference Learning (TD) and Q-learning. Then it touches on 2 other cutting edge topics, Active learning and Bayesian Optimization. You will also learn one key issue in AI planning, explore vs. exploit.

Game Theory

The module ends with the other major topic on Game Theory. This is arguably the most interesting topic of all. For those intellectually inclined, you probably know about the famous Prisoner’s Dilemma. That is just one of the many interesting things in game theory. In this module, you will learn more in-depth knowledge on game theory, from an AI perspective. I actually enjoyed this topic so much that I wrote a post on applying some of its concepts to real life.

Advice

• Look for notes on the Internet for concepts you do not understand. There are many notes on classical topics covered in this module, each providing a slightly different angle to approach and understand the concepts. If you don’t understand one explanation, you can always try another one.
• Search on the Internet for practice questions and tutorial questions. There are only a few variations of the MDP/POMDP or game theory set-up. Practice makes it very easy to score well on those questions. The reverse is also true if you don’t do practice.
• Take with you friends who have a strong interest in research if you are not such a person, the project is no joke.