Robotics: Science and Systems XV

Predicting Human Interpretations of Affect and Valence in a Social Robot

David McNeill, Casey Kennington


As the adoption of robots becomes widespread across more industries and domains, those robots will be placed in new contexts where they will interact with people who do not understand how they work. The consequences of such a disparity can already be seen in how people assign anthropomorphic characteristics to those robots, despite what what the robot designers may have intended. In this paper, we seek to understand how people interpret a social robot's performance of an emotion, what we term 'affective display,' and the positive or negative valence of that affect. To this end, we tasked annotators with observing the Anki Cozmo robot perform its over 900 pre-scripted behaviors and labeling those behaviors with 16 possible affective display labels (e.g., interest, boredom, disgust, etc.). In our first experiment, we trained a neural network to predict annotated labels given multimodal information about the robot's movement, face, and audio. The results suggest that pairing affects to predict the valence between them is more informative, which we confirmed in a second experiment. Both experiments show that certain modalities are more useful for predicting displays of affect and valence. For our final experiment, we generated novel robot behaviors and tasked human raters with assigning scores to valence pairs instead of applying labels, then compared our model's predictions of valence between the affective pairs and compared the results to the human ratings. We conclude that robot designers and researchers cannot assume that people will perceive affect or valence as designed, and make several suggestions for directions of future work.



    AUTHOR    = {David McNeill AND Casey Kennington}, 
    TITLE     = {Predicting Human Interpretations of Affect and Valence in a Social Robot}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2019}, 
    ADDRESS   = {FreiburgimBreisgau, Germany}, 
    MONTH     = {June}, 
    DOI       = {10.15607/RSS.2019.XV.041}