Learning policies for abstract state spaces

Abstract

Applying Q-Learning to multidimensional, real-valued state spaces is time-consuming in most cases. In this article, we deal with the assumption that a coarse partition of the state space is sufficient for learning good or even optimal policies. An algorithm is presented which constructs proper policies for abstract state spaces using an incremental procedure without approximating a Q-function. By combining an approach similar to dynamic programming and a search for policies, we can speed up the learning process. To provide empirical evidence, we use a cart-pole system. Experiments were conducted for a simulated environment as well as for a real plant. © 2005 IEEE.