Grand Challenge — Modeling the Dynamics of Multimodal Communication

Communication is rarely, if ever, unimodal: we use words, gestures, facial movements, and more to make and convey meaning in social interaction. The grand challenge for this session was to find analytical approaches that integrate across these channels and model their temporal interdependencies.

The discussion was grounded in two empirical studies of dyadic gaze coordination, used as a basis for describing how such techniques can integrate across multiple channels of communication and pairwise across interactants:

Richardson, D. C., & Dale, R. (2005) — cross-recurrence quantification analysis (CRQA).

Andrist, S., et al. (2015) — epistemic network analysis (ENA).

To make sure everyone had read at least one paper, the group split by surname: A–L read Richardson & Dale (2005), and M–Z read Andrist et al. (2015).