Iana Lin — living systems

Overview

Mountain and Tree Flip Image

Part 0: Text-to-Image (DeepFloyd)

Random seed: 1024

Prompts:

hands using a figure skate as a knife to cut into a hyper-realistic raw steak-shaped cake
an overhead drone-scan style view of an abandoned ice-skating rink at dawn, the ice cracked and reflecting the first light, faint figure skating loops etched on it, muted pastel palette
a translucent hawk soaring over mountains, its wings filled with floating code snippets instead of feathers
a mixed-media style portrait of a girl running statistical regressions on a foggy mountain trail, numbers dissolving into mist

Reflection: More inference steps improved quality; long/abstract prompts introduced more mistakes/strangeness.

Steak prompt 30 steps — Steak cake (30, 80 steps)

Steak prompt 80 steps — Steak cake (30, 80 steps)

Rink prompt 50 steps — Abandoned rink (50, 100 steps)

Rink prompt 100 steps — Abandoned rink (50, 100 steps)

Hawk prompt 50 steps — Hawk (50, 100 steps)

Hawk prompt 100 steps — Hawk (50, 100 steps)

Runner prompt 20 steps — Runner (20, 50 steps)

Runner prompt 50 steps — Runner (20, 50 steps)

1.1 Forward Process (Adding Noise)

Increasing noise on the Campanile via the forward diffusion process.

Gaussian blur struggles to recover structure from heavily noised inputs.

1.3 One-Step Denoising

Single UNet denoise pass recovers detail but degrades as noise increases.

1.4 Iterative Denoising

Strided timesteps progressively denoise toward the clean estimate.

One-step at heavy noise — One-step (t≈990)

1.5 Diffusion Sampling

Unconditional samples from noise (prompt: "a high quality photo").

1.6 Classifier-Free Guidance

CFG boosts fidelity; sampling with guidance scale > 1.

1.7 SDEdit / Image-to-Image

Denoising with different start noise levels (higher = less edits).

1.7.1 Editing Web & Hand-Drawn Inputs

Progression from low to high noise starts

1.7.2 Inpainting

Filling masked regions while preserving context. From left to right: Original, Mask, Hole, Inpainted.

1.7.3 Text-Conditional Translation

Guided edits from different noise levels; Lower noise looks more like text, higher noise looks more lke image.
Prompt: "A pixel art style image, with high pixel density"

1.8 Visual Anagrams

Images that flip into a different concept when rotated 180°. Each row shows the original orientation and flipped version at different resolutions.

64×64 resolution

Man and campfire flip — Old man ↔ Campfire

Mountain and tree flip — Mountain ↔ Oak tree

256×256 resolution

Campfire — An oil painting of people around a campfire

Melted clocks — An oil painting of a pile of melted clocks

Crying face — An oil painting of a surreal crying face

Mountain — A water color drawing of a mountain

Oak tree — A water color drawing of an oak tree

1.9 Hybrid Images

Combining low- and high-frequency noise estimates to blend two prompts.

Palm × Map hybrid

Balloon × Pig hybrid

Landscape × Bookshelf hybrid

Part B: Training a Diffusion Model from Scratch

Implementing and training UNet-based denoising and flow matching models on MNIST.

Part 1.2: Training a UNet-based Denoiser

Visualizing the noising process and training a UNet to denoise MNIST digits at σ = 0.5.

Noising process at different σ levels

Epoch 1

Epoch 5

Training loss curve

Part 1.2.2: Out-of-Distribution Testing

Testing the denoiser on varying noise levels it wasn't trained on.

Performance across different σ values

Part 1.2.3: Denoising Pure Noise

Training a denoiser to denoise pure noise. The patterns observed in the generated outputs seem to resemble an average of the MNIST digits stacked on top of each other This is likely because there is no prior information about the digits in each noise sample. Thus, to minimize loss, the denoiser has to learn the average of the digits.