Grand Challenge — Wearable Facial Multimodal Sensing: Emteq Showcase

Understanding social interaction requires the ability to sense and map behaviour in ways that are unobtrusive, while also capturing the multimodal nature of social signals and context. In this session, we welcome Emteq Labs, who have been developing a facial wearable sensing device designed to track behaviour through facial optomyography, while also integrating contextual data via a scene camera. Such approaches open exciting opportunities for both research and application. The focus on facial wearables is especially timely, as major players in human–computer interaction (e.g. Meta, Apple) are investing heavily in wearable computing platforms with capabilities for capturing and simulating social behaviour, and for augmenting human–machine and human–human interactions in VR and AR.

Abstract

The boundary between the brain’s inner world — its thoughts, emotions, and cognitive processes — and the external world of perception, interaction, and behaviour is a critical frontier for both psychology and computer science. Emteq Labs provides a novel interface at this boundary, offering a real-time, wearable system that captures affective and cognitive responses through facial muscle activity, head movement, and other physiological signals. By integrating these signals with contextual information from an onboard camera, OCOsense enables a dynamic mapping of internal states to external experiences. This allows for fine-grained analysis of how users emotionally and cognitively engage with their surroundings, offering valuable insights into attention, stress, mood, and behavioural patterns. The ability to synchronise real-time affective sensing with environmental stimuli opens up significant applications across psychological and neuroscientific research, product testing, human–computer/robot interaction, behaviour monitoring, and therapy.

The interactive discussion covered multimodal data integration; technical and ethical challenges; data, analytics, and interpretation; and future directions.

Speakers

Dr Charles Nduka is Chief Science Officer and founder of Emteq. He studied medicine at Oxford and Imperial College and is an NHS Consultant Plastic and Reconstructive Surgeon in London and Sussex. His research focuses on facial expressions, emotion measurement, mental health, and rehabilitation, with awards from the Wellcome Trust, Innovate UK, NIHR, and Horizon 2020. Passionate about health technology, he invented facial optomyography in 2017 for low-power, all-day facial sensing and biofeedback, and founded Brighton-based Emteq Labs to scale this technology.

Professor Hristijan Gjoreski is the Chief Technology Officer and Chief Data Scientist at Emteq Labs, and Associate Professor at the Ss. Cyril and Methodius University of Skopje. He is a recognised expert in applied artificial intelligence and machine learning, specialising in algorithms for e-health, wearable computing, activity recognition, and affective computing — enabling new ways to study, understand, and support human behaviour.