Multi‐frame based adversarial learning approach for video surveillance

Abstract

Foreground-background segmentation (FBS) is one of the prime tasks for automated video-based applications like traffic analysis and surveillance. The different practical scenarios like weather degraded videos, irregular moving objects, dynamic background, etc., make FBS a challenging task. The existing FBS algorithms mainly depend on one of the three different factors, namely (1) complicated training process, (2) additionally trained modules for other applications, or (3) neglect the inter-frame spatio-temporal structural dependencies. In this paper, a novel multi-frame-based adversarial learning network is proposed with multi-scale inception and residual module for FBS. As, FBS is a temporal enlightenment-based problem, a temporal encoding mechanism with decreasing variable intervals is proposed for the input frame selection. The proposed network comprises multi-scale inception and residual connection-based dense modules to learn prominent features of the foreground object(s). Also, feedback of the estimated foreground map of previous frame is utilized to exhibit more temporal consistency. Learning of the network is concentrated in different ways like cross-data, disjoint, and global training-testing for FBS. The qualitative and quantitative experimental analysis of the proposed approach is done on three benchmark datasets for FBS. Experimental analysis on three benchmark datasets proves the significance of the proposed approach as compared to state-of-the-art FBS approaches