# Robotics: Science and Systems I

### Robot Planning in Partially Observable Continuous Domains

*Josep M. Porta, Matthijs T. J. Spaan, Nikos Vlassis*

**Abstract:** We present a value iteration algorithm for learning
to act in Partially Observable Markov Decision Processes (POMDPs) with
continuous state spaces. Mainstream POMDP research focuses on the
discrete case and this complicates its application to, e.g., robotic
problems that are naturally modeled using continuous state spaces. The
main difficulty in defining a (belief-based) POMDP in a continuous
state space is that expected values over states must be defined using
integrals that, in general, cannot be computed in closed from. In this
paper, we first show that the optimal finite-horizon value function
over the continuous infinite-dimensional POMDP belief space is
piecewise linear and convex, and is defined by a finite set of
supporting α-functions that are analogous to the α-vectors
(hyperplanes) defining the value function of a discrete-state
POMDP. Second, we show that, for a fairly general class of POMDP
models in which all functions of interest are modeled by Gaussian
mixtures, all belief updates and value iteration backups can be
carried out analytically and exact. A crucial difference with respect
to the &alhpa;-vectors of the discrete case is that, in the continuous
case, the α-functions will typically grow in complexity (e.g.,
in the number of components) in each value iteration. Finally, we
demonstrate PERSEUS, our previously proposed randomized point-based
value iteration algorithm, in a simple robot planning problem with a
continuous domain, where encouraging results are observed.

**Bibtex:**

@INPROCEEDINGS{ Porta-RSS-05, AUTHOR = {Josep M. Porta and Matthijs T. J. Spaan and Nikos Vlassis}, TITLE = {Robot Planning in Partially Observable Continuous Domains}, BOOKTITLE = {Proceedings of Robotics: Science and Systems}, YEAR = {2005}, ADDRESS = {Cambridge, USA}, MONTH = {June}, DOI = {10.15607/RSS.2005.I.029} }