Grand Challenge — Studying Social Interactions with Real-Time Voice and Face Transformations

It has been remarkably difficult for social interaction research to study interactive contexts. Studies analysing “free conversation” recordings can identify features that covary with behaviour, but only provide correlational findings. Experiments that control individual factors — using, for example, scripted interactions or virtual avatars — can support causal inference, but are limited in ecological validity.

To overcome these limitations, we have been building a new experimental platform called DuckSoup. DuckSoup is a videoconferencing platform, like Zoom or Skype, but one where researchers can manipulate participants’ facial and vocal attributes in real time using voice and face transformation algorithms — for example artificially increasing one participant’s smiles, or changing the emotional tone of their voice. This gives researchers the ability to covertly control social signals during interactions. The session brainstormed what would be possible with this platform, at an early stage of its testing and development.