Abstract:
Generative adversarial networks (GANs) are among
the most popular deep learning models for learning complex
data distributions. However, training a GAN is known to be a
challenging task. This is often attributed to the lack of correlation
between the training progress and the trajectory of the generator
and discriminator losses and the need for the GAN’s subjective
evaluation. A recently proposed measure inspired by game theory
- the duality gap, aims to bridge this gap. However, as we
demonstrate, the duality gap’s capability remains constrained due
to limitations posed by its estimation process. This paper presents
a theoretical understanding of this limitation and proposes a
more dependable estimation process for the duality gap. At the
crux of our approach is the idea that local perturbations can
help agents in a zero-sum game escape non-Nash saddle points
efficiently. Through exhaustive experimentation across GAN
models and datasets, we establish the efficacy of our approach in
capturing the GAN training progress with minimal increase to the
computational complexity. Further, we show that our estimate,
with its ability to identify model convergence/divergence, is a
potential performance measure that can be used to tune the
hyperparameters of a GAN.
