System Atlas
the atlas
AI · AI & LLMs8 minDeep dive

How Generative Adversarial Networks work

Two neural networks locked in a forger-versus-detective duel until the fakes become indistinguishable.

the gist

A GAN pits a generator that turns random noise into fake data against a discriminator that tries to tell real from fake. Trained as a minimax game, each network pushes the other to improve until the generator's output is indistinguishable from real data. The training is unstable and prone to mode collapse, but GANs power image synthesis, super-resolution, and more.

ANALOGY // A counterfeiter (generator) prints fake bills while a detective (discriminator) learns to spot them; each forces the other to sharpen, until the counterfeits are so good the detective can only guess.
living diagram01 / 06
Random noiselatent vector zGeneratornoise → fakeReal datatraining setDiscriminatorreal vs fakeReal / fake?probabilityAdversarial lossbackprop to both
01

Noise into a fake

The generator takes a vector of random noise and transforms it, through layers of a neural network, into a synthetic sample such as an image. Crucially it never looks at real data directly — it only ever produces candidates. Its whole task is to learn a mapping from meaningless noise to outputs that look like they came from the real distribution.

02

The discriminator judges

The discriminator is a binary classifier that receives a sample — sometimes real from the dataset, sometimes fake from the generator — and outputs the probability that it is real. It is trained on labeled examples of both, so it learns the features that separate genuine data from forgeries. Its job is to be a sharp, well-calibrated detective.

03

The adversarial game

The two networks are trained against each other in a minimax game: the discriminator tries to maximize its accuracy at spotting fakes while the generator tries to minimize it by fooling the detective. The generator's gradient comes entirely from the discriminator's judgment — it improves by exploiting whatever the detective currently gets wrong. This adversarial pressure is the whole engine of learning.

04

Backprop and update

Each round, the discriminator updates to better separate real from fake, then the generator updates using the discriminator's feedback to make its fakes more convincing. The two are locked in a loop, alternately improving, with each one's progress raising the bar for the other. Balancing their learning rates so neither overpowers the other is the central difficulty of training.

05

Reaching equilibrium

In the ideal outcome the generator's samples become statistically indistinguishable from real data, so the discriminator can do no better than a coin flip and outputs one-half everywhere. At this theoretical equilibrium the generator has effectively learned the true data distribution. In practice training is stopped when the samples look good enough, because perfect equilibrium is hard to reach.

06

Mode collapse & uses

The notorious failure is mode collapse, where the generator finds a few outputs that reliably fool the discriminator and produces only those, ignoring the diversity of real data. Careful architectures, loss functions, and training tricks mitigate it. When they work, GANs deliver photorealistic image synthesis, super-resolution, style transfer, and synthetic training data.

the full picture
01

The idea — learning by competition

Most neural networks learn from a teacher: you show them labeled examples and correct their mistakes. Generative Adversarial Networks, introduced by Ian Goodfellow and colleagues in 2014, use a radically different trick — they learn by making two networks compete. There is no fixed teacher; instead each network's opponent *is* its teacher, and the teaching gets harder exactly as the student improves.

The setup is a forger versus a detective. One network, the generator, tries to produce fake data (say, images of faces) convincing enough to pass as real. The other, the discriminator, tries to catch the fakes. The generator never sees the real data directly; its only signal is whether it fooled the detective. As the discriminator gets sharper, the generator is forced to get better, and as the generator gets better, the discriminator must sharpen again. This escalating arms race is the entire learning mechanism, and its elegance is why GANs reshaped generative modeling.

02

The two networks

The pieces are simple to describe individually.

  • The generator G takes a vector of random noise z — a sample from a simple latent distribution — and transforms it, through layers of a neural network, into a synthetic output like an image. It is a mapping from meaningless noise to structured, realistic-looking data. Different noise vectors produce different outputs, which is what lets a trained generator create endless novel samples.
  • The discriminator D is an ordinary binary classifier. It takes a sample and outputs a single number: the probability that the sample is real rather than generated. During training it is fed a mix of real examples from the dataset (labeled real) and fakes from the generator (labeled fake), and it learns the features that distinguish the two.

Neither is exotic on its own — the magic is entirely in how they are trained against each other rather than in either architecture.

03

The minimax game and the training loop

Formally, a GAN is a minimax game between the two networks over a single value function. The discriminator wants to *maximize* the probability it assigns correct labels — real to real, fake to fake — while the generator wants to *minimize* that same quantity by producing fakes the discriminator scores as real. One network's gain is the other's loss.

Training alternates:

  • Train the discriminator. Feed it a batch of real samples and a batch of fakes, and update its weights to better separate the two. Its loss drops as it gets better at catching forgeries.
  • Train the generator. Feed noise through the generator, pass the fakes to the (now-frozen) discriminator, and backpropagate through the discriminator into the generator so its outputs score higher as "real." Notice the generator's gradient comes *entirely through the discriminator* — it improves by exploiting whatever the detective currently gets wrong.

Repeat, and both climb. The delicate part is balance: if the discriminator becomes too strong it rejects everything and the generator gets no useful gradient; if it is too weak it gives no meaningful signal. Keeping the two roughly matched — via learning rates, architecture choices, and alternate-loss formulations — is the core practical challenge.

04

Equilibrium and failure modes

In theory the game has a beautiful fixed point. As the generator improves, its output distribution approaches the real data distribution, and once they match, the discriminator genuinely *cannot* tell real from fake — the best it can do is guess, outputting a probability of one-half everywhere. At that Nash equilibrium the generator has effectively learned the true data distribution, which is exactly what you wanted.

Reality is messier, and GANs are notoriously unstable to train.

  • Mode collapse is the signature failure: the generator discovers a small set of outputs that reliably fool the current discriminator and produces only those, ignoring the full variety of the real data. You get one convincing face over and over instead of a diverse population.
  • Non-convergence and oscillation happen when the two networks chase each other without settling, losses swinging rather than stabilizing.
  • Vanishing gradients occur when the discriminator wins too decisively and stops giving the generator anything to learn from.

Much of GAN research — Wasserstein loss, gradient penalties, spectral normalization, progressive growing — exists specifically to tame these instabilities.

05

What GANs are good for

When they train well, GANs are exceptional at producing sharp, realistic samples, and that has driven a wave of applications.

  • Image synthesis — generating photorealistic faces, objects, and scenes that never existed (the "this person does not exist" demos were StyleGAN).
  • Super-resolution — upscaling a low-resolution image into a plausible high-resolution one by hallucinating convincing detail.
  • Image-to-image translation and style transfer — turning sketches into photos, day into night, or one artistic style into another (as with CycleGAN).
  • Data augmentation — synthesizing extra training examples for domains where real labeled data is scarce, such as medical imaging.

It is worth situating GANs among their rivals: diffusion models have since overtaken them for many high-end image-generation tasks because they train more stably and cover diversity better, while GANs retain an edge where fast, single-pass generation matters. But the core GAN idea — training by adversarial competition — was a genuine conceptual leap, and adversarial objectives now appear far beyond image generation, anywhere you can pit a maker against a critic.

what to remember
01

The generator maps random noise to synthetic samples; it never sees real data directly.

02

The discriminator is a binary classifier scoring how likely a sample is real versus fake.

03

Training is a minimax game: the generator minimizes what the discriminator maximizes.

04

Their competition drives both toward an equilibrium where fakes are indistinguishable from real data.

05

Mode collapse — the generator producing only a few outputs — is the classic failure mode.

06

Uses include photorealistic image synthesis, super-resolution, style transfer, and data augmentation.

check yourself

Quick quiz

Q1

What is the generator's input during training?

Q2

What is mode collapse?

follow the thread