Robotics: Science and Systems VIII

Variational Bayesian Optimization for Runtime Risk-Sensitive Control

Scott Kuindersma, Roderic Grupen, Andrew Barto

Abstract:

We present a new Bayesian policy search algorithm suitable for problems with policy-dependent cost variance, a property present in many robot control tasks. We extend recent work on variational heteroscedastic Gaussian processes to the optimization case to achieve efficient minimization of very noisy cost signals. In contrast to most policy search algorithms, our method explicitly models the cost variance in regions of low expected cost and permits runtime adjustment of risk sensitivity without relearning. Our experiments with artificial systems and a real mobile manipulator demonstrate that flexible risk-sensitive policies can be learned in very few trials.

Download:

Bibtex:

  
@INPROCEEDINGS{Kuindersma-RSS-12, 
    AUTHOR    = {Scott Kuindersma AND Roderic Grupen AND Andrew Barto}, 
    TITLE     = {Variational Bayesian Optimization for Runtime Risk-Sensitive Control}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2012}, 
    ADDRESS   = {Sydney, Australia}, 
    MONTH     = {July},
    DOI       = {10.15607/RSS.2012.VIII.026} 
}