SBI-DHGR: Skeleton-based intelligent dynamic hand gestures recognition

Abstract

Abstract: Hand gesture recognition (HGR) plays a significant role in interpreting the meaning of sign language, human–computer interaction, and robot control. This paper proposes a real-time skeleton-based intelligent dynamic hand gesture recognition (SBI-DHGR) approach, which comprises three modules: Palm Centroid (PC), Data augmentation with the Tenet Effect, and Deep learning architecture. The palm Centroid (PC) module is introduced to identify the proposed palm center joint. Data augmentation with a tenet effect module is designed to improve the CNN model’s generalizability. Further, a novel deep learning model is proposed for temporal 3D-HGR by exploiting the capabilities of a multi-channel convolutional neural network (CNN) and long short-term memory (LSTM) recurrent network. The multi-channel CNN is introduced to learn the cardinal positions of hand joints by capturing the low, average, and high-level features. LSTM is embedded to learn the temporal characteristics of hand joints. The effectiveness of the SBI-DHGR framework is evaluated over five challenging datasets: SHREC-14, SHREC-28, DHG-14, DHG-28, and FPHA, by adopting person-dependent and person-independent validation setups.