Master's Thesis Project - Computer Vision and Human Behavior Analysis

Human communication is a major field of study in psychology and social sciences. Topics such as emergent leadership and group dynamics are commonly studied cases when referring to group settings. Experiments regarding group settings are usually analyzed in conversational and collaborative tasks environments in order to study the communication process in small groups.

Former studies' methods involve human analysis and manual annotation of others' behaviors in communication settings. Later studies try to replace time consuming and failure prone annotations by resorting to computational methods.

Having a custom, newly-gathered audiovisual dataset, from an experiment conducted by the Department of Education and Psychology of the University of Aveiro, a multidisciplinary group from the same institution with members from psychology and engineering backgrounds, took the initiative to create computational methods in order to facilitate the analysis of the collected data.

For that purpose, this work presents a multimodal computational framework using state-of-the-art methods in computer vision, capable of enriching image data with annotations of a broad range of nonverbal communication aspects, both at an individual and group levels, thus facilitating the study of nonverbal communication and group dynamics.

This works contributes to the community by presenting methods to directly increase human knowledge about the human communication process, involving data transformation processes in order to transform raw feature data into humanly understandable meanings and a visualization tool capable of visualizing such methods applied to the input data.