Training diffusion models is computationally heavy, especially for frontier models, and beginners often struggle to find the opportunity to run the whole pipeline. MiniDog is a simple yet useful codebase, where pretraining and fine-tuning of text-to-image (T2I) models can be performed in about an hour and 15 minutes, respectively, on four 3090 GPUs. It targets advanced undergraduate and graduate courses and assumes linear algebra, calculus, probability, and deep learning. On completing the material, beginners can:

§Build and pretrain: a text-to-image flow-matching pipeline on consumer GPUs.
§Explain what changes FID: how the tokenizer, caption length, and representation alignment change FID.
§Fine-tune and evaluate: fine-tune the model and evaluate it with human-preference models.

Training Recipe Pipeline Results Method Comparisons

Click to jump to each section.

The MiniDog Resource

Data. Pretraining uses 26,000 recaptioned ImageNet images of 20 dog breeds. Supervised fine-tuning (SFT) uses 2,000 high-quality images of the same breeds that we generate with FLUX.1-schnell from text prompts written in the same structure as the real-photo captions. Evaluation uses 500 held-out prompts. All three datasets are released on Hugging Face.

Pipeline. We adopt the DiffusionBench framework. During pretraining, the image caption passes through Qwen3-0.6B to obtain text tokens, and the paired image is fed to a VAE and an external vision encoder. Noise is added to the image latent. Timestep and text tokens are concatenated with the noisy latents into one sequence (in-context conditioning), and this sequence is fed into the diffusion transformer. The vision perception tokens are aligned with intermediate diffusion features at Block 4 of 12, and the flow-matching loss is also applied. At the SFT stage, we turn off the alignment loss and only use the flow-matching loss.

Diagram of the pretraining recipe. A caption goes through a frozen Qwen3-0.6B text encoder, the image through a frozen E2E-INVAE tokenizer and a frozen EUPE encoder. Text embeddings, a time token and noisy latents form one sequence for a trainable 12-block LightningDiT. Block 4 is aligned to the EUPE features, and the output is trained with the flow-matching loss. — **T2I pretraining recipe.** Frozen Qwen3, E2E-INVAE and EUPE ViT-B/16 produce the text embeddings, the latents (noised with ε) and the alignment targets. LightningDiT is pretrained with the flow-matching loss and the iREPA loss on Block 4.

**T2I pretraining recipe.** Frozen Qwen3, E2E-INVAE and EUPE ViT-B/16 produce the text embeddings, the latents (noised with ε) and the alignment targets. LightningDiT is pretrained with the flow-matching loss and the iREPA loss on Block 4.

Methods supported:

Architecture: latent diffusion with a 12-block transformer (22.06M parameters) and in-context conditioning
VAE: E2E-INVAE, E2E-VAVAE
Representation alignment: REPA, iREPA
Vision encoders: DINOv2, DINOv3, PE-Spatial, EUPE
Fine-tuning: supervised fine-tuning and reward-model-based scoring

Evaluation metrics. We measure generation quality with gFID, using all 26k real pretraining images as the reference set and 26k images generated from the same pretraining prompts. We measure tokenizer reconstruction with PSNR, SSIM, LPIPS and rFID. To assess the effect of SFT, we use two human-preference models, HPSv2 and PickScore, averaged over 500 images that the pretrained and the SFT checkpoints each generate from the 500 held-out prompts.

Training Recipe

Unless otherwise stated, we pretrain for 200 epochs (~20k steps) and fine-tune for 100 epochs (~700 steps). Click below to see every hyperparameter of the final recipe.

Show all hyperparameters of the final recipe

**Hyperparameters of the final recipe.** Configs: `pretrain_e2e-invae_128tok_mse_irepa_eupe.yaml` and `sft_e2e-invae_128tok_mse_norepa.yaml`.
	Pretraining	SFT
Architecture
Backbone	LightningDiT
Input dim.	16×16×32
Patch size	1
Num. layers	12
Hidden dim.	384
Num. heads	6
Timestep / text tokens	4 / 128
Text encoder	Qwen3-0.6B (frozen)
Tokenizer	E2E-INVAE (frozen)
Alignment (final recipe)
Target representation	EUPE ViT-B/16	–
Method	iREPA	–
Distance d	mean squared error	–
λ	0.5	–
Alignment depth	4	–
Projector g_φ	3×3 conv, 384→768, padding 1	–
Spatial norm. γ, ε	0.7, 10⁻⁶	–
Alignment (compared in the vision-encoder table)
Target representations	DINOv2, DINOv3, PE-Spatial, EUPE	–
Methods	REPA (linear, 384→768), iREPA	–
Distances d	mean squared error, cosine	–
Optimization
Training data	26k images	2k images
Epochs	200 (~20k steps)	100
Batch size	256	256
Optimizer	AdamW	AdamW
Learning rate	10⁻⁴, then linear decay	5×10⁻⁵, constant
(β₁, β₂)	(0.9, 0.999)	(0.9, 0.999)
Weight decay	0	0
Gradient clipping	1.0	1.0
EMA decay	0.9995	0.995
Condition dropout	0.1	0.1
Precision	bf16	bf16
Interpolants and sampling
α_t, σ_t	1−t, t
Training objective	v-prediction
Timestep sampling	logit-normal(0,1), shift √2
Sampler	Euler (ODE), 50 steps
Guidance (CFG scale)	6.0

Computing resources:

GPUs: 4 of the 8 NVIDIA RTX 3090 GPUs (24 GB, 350 W) in a single server, connected over PCIe 4.0 without NVLink
CPU and memory: two AMD EPYC 7343 CPUs (32 cores in total), 256 GB of RAM, and the cached inputs on a local NVMe SSD
Software: Ubuntu 24.04, NVIDIA driver 590.44, Python 3.13, PyTorch 2.10.0 with CUDA 12.8, cuDNN 9.10, NCCL 2.27, and Triton 3.6; pyproject.toml and uv.lock pin every package version
Parallelism: PyTorch DistributedDataParallel over the 4 GPUs (64 images per GPU)

Results

Our best recipe

Our best pretraining recipe is to use E2E-INVAE as the tokenizer, 128-token captions, iREPA for acceleration with a mean squared error loss, and EUPE as the external vision encoder. This recipe yields gFID = 8.32 at a resolution of 256×256. We then apply standard SFT, which gives HPSv2 = 0.2427 and PickScore = 19.61. Samples after SFT are shown at the top of this page.

Efficiency

Using four 3090 GPUs, pretraining for 200 epochs takes 55 minutes, while SFT for 100 epochs takes 15 minutes. The image and text tokens are precomputed and cached before training.

MiniDog also runs on a single 3090 GPU. Accumulating gradients over four micro-batches of 64 images keeps the batch of 256 images and the training schedule of the four-GPU run, and pretraining then takes about 3 hours, excluding the in-training FID evaluation.

Method Comparisons

Each comparison changes one setting and keeps the others fixed. In the tables, green marks the setting used in the final recipe, and bold marks the best value.

Comparing E2E-INVAE and E2E-VAVAE

Both VAEs, trained by the REPA-E method, are publicly available. To compare VAEs, both reconstruction and diffusability are important. We first compare their reconstruction performance: across the four metrics, E2E-INVAE is consistently better. We then compare how they support diffusion model training, where E2E-INVAE also yields lower gFID scores, that is, better diffusability.

**VAE reconstruction comparison.** We report four metrics for the E2E-INVAE and E2E-VAVAE.
Tokenizer	PSNR↑	SSIM↑	LPIPS↓	rFID↓
E2E-INVAE	28.42	0.792	0.039	1.14
E2E-VAVAE	27.59	0.753	0.044	1.39

FID against training steps from 10k to 20k. E2E-VAVAE falls from about 27 to 12.23; E2E-INVAE falls from about 16.5 to 8.32 and stays lower throughout. — **VAE diffusability comparison.** Pretraining gFID with EUPE, iREPA, and 128 tokens.

Long caption vs. short caption

For each dog image, we provide a short caption (64 tokens maximum) and a long caption (128 tokens maximum). We compare them using E2E-INVAE, with iREPA, the EUPE encoder and the MSE loss. Training with long captions yields lower gFID, which indicates that long captions allow pretraining to converge faster than short captions.

**Comparing the FIDs of short and long captions in the pretraining stage.**
FID↓	64 tokens	128 tokens
E2E-INVAE	8.54	8.32

REPA vs. iREPA

We compare REPA, iREPA and a baseline without REPA. REPA clearly accelerates convergence of the baseline. Moreover, when using the EUPE encoder, iREPA converges faster than REPA.

FID against training steps from 10k to 20k for three runs. Without REPA, FID falls from about 34.6 to about 11.5; with REPA from about 19.7 to 9.71; with iREPA from about 16.5 to 8.32. — **Comparing REPA, iREPA, and no REPA.** We use E2E-INVAE, EUPE encoder, 128 tokens, and MSE loss.

Comparing different vision encoders

DINOv2 performs the best for REPA, while EUPE performs the best for iREPA. While REPA is slightly better under the PE-Spatial encoder, the overall best performance is achieved with iREPA and EUPE. The mean squared error loss is superior to the cosine similarity loss for both REPA and iREPA.

**Comparing external vision encoders, REPA vs. iREPA, and MSE vs. cosine similarity.** We use E2E-INVAE and 128 prompt tokens. λ = 0.5.
Target representation	REPA FID↓	iREPA FID↓
Squared error
DINOv2 ViT-B/14	8.80	8.68
PE-Spatial B/16	9.56	9.65
EUPE ViT-B/16	9.71	8.32
DINOv3 ViT-B/16	10.12	8.46
Cosine
DINOv2 ViT-B/14	9.14	8.89
PE-Spatial B/16	9.89	9.99
EUPE ViT-B/16	10.20	8.37
DINOv3 ViT-B/16	10.32	9.01

Benefit of supervised fine-tuning

We compare our pretrained model and the SFT model with two pretrained reward models, which generally reflect how much people would like an image given its prompt. The HPSv2 and PickScore of the SFT model are consistently higher than those of the pretrained model, mainly because of the high-quality SFT data. FID (not shown) of the SFT model is higher than that of the pretrained model, because the high-quality SFT data pulls the model away from the pretraining distribution. The samples at the top of this page are cherry-picked.

**Human-preference scores of the pretrained and SFT checkpoints.**
	HPSv2↑	PickScore↑
Pretrained	0.1625	18.42
SFT	0.2427	19.61

Conclusion

MiniDog does not build a comprehensive T2I model. Instead, it is a minimal project that focuses on dog image generation for the purpose of democratizing T2I training. Under our configuration, pretraining takes only about an hour on four 3090 GPUs and yields an FID of 8.32 at a resolution of 256×256. Users can spend another 15 minutes on supervised fine-tuning, which improves image quality as judged by pretrained reward models. The entire project, including data, models, and code, has been open-sourced, with the hope of allowing researchers to get started and evaluate their ideas.

MiniDog

Dog-Breed Text-to-image Flow Matching from Pretraining to Fine-Tuning on 4 Consumer GPUs