StyleGAN

Most generators stir $z$ into every layer at the same time. StyleGAN split it apart: first map $z$ to an intermediate $w$, then inject $w$ at every resolution level through AdaIN. That single change untangled the latent space and made photorealistic faces a solved problem for seven years running.

Type: Build Languages: Python Prerequisites: Phase 8 · 03 (GANs), Phase 4 · 08 (Normalization), Phase 3 · 07 (CNNs) Time: ~45 minutes

The Problem

A DCGAN maps $z$ to an image through a stack of transposed convolutions. The problem: $z$ controls everything — pose, lighting, identity, background — entangled together. Move along one axis of $z$, all four change. You cannot ask the model "same person, different pose" because the representation does not factor that way.

Karras et al. (2019, NVIDIA) proposed: stop feeding $z$ directly into conv layers. Feed a constant 4×4×512 tensor as the network input. Learn an 8-layer MLP that maps $z \in Z \to w \in W$. Inject $w$ at every resolution via adaptive instance normalization (AdaIN): normalize each conv feature map, then scale and shift by affine projections of $w$. Add per-layer noise for stochastic detail (skin pores, hair strands).

The result: $W$ has roughly orthogonal axes for "high-level style" (pose, identity) vs "fine style" (lighting, color). You can swap styles between two images by using image A's $w$ for the low-resolution levels and image B's $w$ for the high. This unlocked editing, cross-domain stylization, and the entire "StyleGAN-inversion" line of research.

The Concept

Mapping network. $f: Z \to W$, an 8-layer MLP. $Z = \mathcal{N}(0, I)^{512}$. $W$ is not forced to be Gaussian — it learns a data-adapted shape.

Synthesis network. Starts from a learned constant 4×4×512. Each resolution block: upsample → conv → AdaIN(w_i) → noise → conv → AdaIN(w_i) → noise. Resolutions double: 4, 8, 16, 32, 64, 128, 256, 512, 1024.

AdaIN.

where $y_{\text{scale}}$ and $y_{\text{bias}}$ come from affine projections of $w$. Normalize per feature map, then restyle. "Style" here is the first- and second-order statistics of the feature map.

Per-layer noise. Single-channel Gaussian noise added to each feature map, scaled by a learned per-channel factor. Controls stochastic detail without affecting global structure.

Truncation trick. At inference, sample $z$, compute $w = \text{mapping}(z)$, then $w' = \hat{w} + \psi \cdot (w - \hat{w})$ where $\hat{w}$ is the mean $w$ over many samples. $\psi < 1$ trades diversity for quality. Almost every StyleGAN demo uses $\psi \approx 0.7$.

StyleGAN 1 → 2 → 3

In 2026 StyleGAN3 remains the default for (a) narrow-domain photorealism at high FPS, (b) few-shot domain adaptation (train on a new dataset with 100 images, freeze mapping), (c) inversion-based editing (find the $w$ that reconstructs a real photo, then edit that $w$). For open-domain text-to-image, it is not the tool — diffusion is.

Interactive figure “gx-stylegan-mapping”: open it on the curriculum site ↗

Build It

code/main.py implements a toy "style-GAN lite" in 1-D: a mapping MLP, a synthesis function that takes a learned constant vector and modulates it with $w$-derived scale/bias, and per-layer noise. It shows that injecting $w$ via affine-modulation matches or beats concatenating $z$ into the generator's input.

Step 1: mapping network

def mapping(z, M):
    h = z
    for i in range(num_layers):
        h = leaky_relu(add(matmul(M[f"W{i}"], h), M[f"b{i}"]))
    return h

Step 2: adaptive instance normalization

def adain(x, w_scale, w_bias):
    mu = mean(x)
    sd = std(x)
    x_norm = [(xi - mu) / (sd + 1e-8) for xi in x]
    return [w_scale * xi + w_bias for xi in x_norm]

Per-feature-map scale and bias come from $w$ via linear projection.

Step 3: per-layer noise

def add_noise(x, sigma, rng):
    return [xi + sigma * rng.gauss(0, 1) for xi in x]

Sigma per-channel is learnable.

Pitfalls

• Droplet artifacts. StyleGAN 1 produced a blobby droplet in the feature maps because AdaIN zeroed out mean. StyleGAN 2's weight demodulation fixes it by scaling the convolution weights instead.
• Texture sticking. StyleGAN 1 and 2 textures followed pixel coordinates, not object coordinates (visible when interpolating). StyleGAN 3's alias-free convolutions fix this with windowed sinc filters.
• Mode coverage. Truncation ψ < 0.7 looks clean but samples from a narrow cone; use ψ = 1.0 if you need diversity.
• Inversion is lossy. Inverting a real photo into $W$ is usually done through optimization or an encoder (e4e, ReStyle, HyperStyle). Results drift over many iterations.

Use It

For product-grade demos where the answer is "photo of a person's face", StyleGAN beats diffusion on inference cost (single forward pass, <10ms on a 4090) and sharpness for the same quality bar.

Ship It

Save outputs/skill-stylegan-inversion.md. Skill takes a real photo and outputs: inversion method (e4e / ReStyle / HyperStyle), expected latent loss, editing budget (how far in $W$ you can move before artifacts), and a list of known-good editing directions (age, expression, pose).

Exercises

1. Easy. Run code/main.py with adain_on=True and adain_on=False. Compare the spread of outputs for a fixed latent vs perturbed latent.
2. Medium. Implement mixing regularization: for a training batch, compute $w_a$, $w_b$, and apply $w_a$ for the first half of synthesis and $w_b$ for the second half. Does the decoder learn disentangled styles?
3. Hard. Take a pretrained StyleGAN3 FFHQ model (ffhq-1024.pkl). Find the $w$ direction that controls "smile" by training an SVM on labelled samples; report how far you can push before identity drifts.

Key Terms

Production note: why StyleGAN still ships in 2026

StyleGAN3 on a 4090 generates a 1024² FFHQ face in under 10 ms — num_steps = 1, no VAE decode, no cross-attention pass. In production terms this is the floor latency for any image generator. A 50-step SDXL + VAE-decode pipeline at the same resolution is ~3 seconds. That is a 300× gap, and for narrow-domain products (avatar services, ID document pipelines, stock face generation) it wins on TCO.

Two operational consequences:

• No scheduler, no batcher. Static batch at the target occupancy is optimal. Continuous batching (essential for LLMs and diffusion) provides zero benefit because every request takes the same FLOPs.
• Truncation ψ is the safety knob. ψ < 0.7 samples from a narrow cone of the mapping network's range. This is the only lever the serving layer has over sample variance. Lower ψ at peak load, raise it for premium users.

Further Reading

• Karras et al. (2019). A Style-Based Generator Architecture for GANs — StyleGAN.
• Karras et al. (2020). Analyzing and Improving the Image Quality of StyleGAN — StyleGAN2.
• Karras et al. (2021). Alias-Free Generative Adversarial Networks — StyleGAN3.
• Tov et al. (2021). Designing an Encoder for StyleGAN Image Manipulation — e4e inversion.
• Sauer et al. (2022). StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets — StyleGAN-XL.
• Huang et al. (2024). R3GAN: The GAN is dead; long live the GAN! — modern minimal GAN recipe.