Hybrid neural networks based approach for holoscopic micro-gesture recognition in images and videos

Abstract

This paper presents an approach for hand based micro-gesture recognition in images and videos as part of the Holoscopic Micro-Gesture Recognition (HoMGR) challenge. The database consists of Holoscopic 3D Micro-Gesture images and videos. The proposed framework is an ensemble of convolutional neural network and deep neural network. The framework performs feature fusion technique on both handcrafted (local phase quantization) and deep features extracted from the neural network, to leverage on complimentary information. The powerful discriminative nature of the fused features has proved beneficial on the given HoMGR challenge data. The experiments show that the proposed approach is effective and outperforms the baseline on the Test set by an absolute margin of 26.67% for images and 2.47% for videos, respectively