Progressive trends inmulti-weather image/video restoration

Abstract

In the domain of computer vision, cameras are integral to various applications including traffic monitoring, scene comprehension, and autonomous navigation. Similarly, satellite imaging plays a vital role in terrestrial surveillance and agronomic assessments. Additionally, underwater autonomous vehicles utilize camera systems to observe marine life. The efficacy of these systems heavily relies on the quality of the visual input they receive. However, environmental factors such as atmospheric disturbances (rain, haze, snow, clouds) and underwater variations (scattering, color shifts) can significantly degrade image visibility. This compromises the quality of the raw visual data and also adversely affects the performance of advanced computer vision algorithms that depend on this data for accurate output. The existing deep learning based methods for weather degraded image restoration have limitations such as single weather applicability, multiple weather specific encoders, data-prior dependency, and limited performance on real-world restoration. To mitigate these limitations, in this work, we have proposed novel methodologies for multi-weather image/video restoration. Firstly, we propose generative adversarial networks based solutions for weather degraded image/video restoration. These solutions consist of (a) progressive subtractive approach, (b) adaptive varying receptive fields, (c) degradation removal and feature refinement streams. Further, we have proposed transformer based solutions for weather degraded image restoration consisting of (a) edge boosting skip connections and memory replay training, (b) spatially attentive deformable transformers, (c) global spatial context aware channel attentive feature propagator, and (d) density aware query modulated block. Also, we have proposed blind diffusion models particularly aimed at restoring complex multi-weather degraded images. Finally, we have proposed a prompted generalized domain translation approach for multi-weather image restoration. Experiments on various synthetic and real-world datasets, along with ablation studies prove the effectiveness of the proposed approaches for multi-weather image/video restoration.