Robot Sound in Interaction : Analyzing and Designing Sound for Human-Robot Coordination

Abstract: Robots naturally emit sound, but we still know little about how sound can serve as an interface that makes a robot’s behavior explainable to humans. This dissertation draws on insights about human practices for coordinating bodily activities through sound, investigating how they could inform robot design. My work builds on three video corpora, involving i) a Cozmo robot in ten family homes, ii) autonomous public shuttle buses in an urban environment, and iii) a teamwork robot prototype controlled by a researcher and interacting with study participants in an experimental setting. I approached the data from two methodological angles, exploring how they can speak to each other: I first carried out an empirical analysis of the video data from an Ethnomethodology and Conversation Analysis (EMCA) perspective, focusing on how humans make sense of robot sound on a moment-by-moment basis in naturally occurring interaction. Subsequently, taking an Interaction Design perspective, I used my video recordings as a design material for exploring how robot sound could be designed in and for real-time interaction. My work contributes to Human-Robot Interaction through detailed studies of robots in the world (rather than in the lab), focusing on how participants make sense of robot sounds. I present a novel framework for designing sound in and for interaction and a prototyping practice that allows practitioners to embed an EMCA stance into their designs. The dissertation contributes to EMCA by describing how members embed autonomous machines into the social organization of activities and how humans treat robots as participants in the interaction. I make a contribution to the development of EMCA hybrid studies by seeking a synthesis between EMCA and robot interaction design.