SEWA project to develop methods for automatic behavioural analysis

The main aim of SEWA is to deploy and capitalise on existing state-of-the-art methodologies, models and algorithms for a machine analysis of facial, vocal and verbal expressions: the analysis results are subsequently adjusted and combined to achieve natural human-centric human-computer interaction (HCI) and computer-mediated face-to-face interaction (FF-HCI).

The envisioned technology is based on research in the cognitive sciences and will yield methods for the automatic analysis of human spontaneous patterns of behavioural cues, including the analysis of mood, affinity and empathy.

Sample applications for the developed technology are personalised recommendations of movie trailers and the automatic suggestion of conversational partners in video chat.

Under this project, which is funded under the European Union’s Horizon 2020 Framework Programme for Research and Innovation, the University of Passau will be collaborating with world-renowned university Imperial College London and two industrial partners, Estonian company RealEyes, and London-based PlayGen. The project has a term of three and a half years.

The research conducted at the University of Passau will focus primarily on the field of highly robust speech and audio analysis in noisy environments.

Furthermore, the researchers at the Chair of Complex and Intelligent Systems will exploit their machine-learning expertise to create self-learning and adaptive multimodal recommendation systems based on audio and visual information.

Finally, the University of Passau will provide support with studies concerning user acceptance and ethical considerations.

Note for editors: Please address your enquiries to the University's Media Relations Section, phone: +49 851 509 1439.