GraphITTI: Atributed Graph-based Dominance Ranking in Social Interaction Videos

Abstract

Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to refor- mulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specifc attributes such as facial gestures, eye gaze, visual attention and speaking pat- terns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spa- tiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.