training-and-dataGuides FastVideo dataset layouts, latent preprocessing, modular YAML training, legacy recipe boundaries, and resource-safe training setup.
Independent GitHub snapshot · Automated classification · Original documentation governs
Use for raw video/caption organization, precomputed latent datasets, preprocessing, fine-tuning, and the new modular trainer.
FastVideo commonly precomputes text embeddings and VAE latents to reduce
training memory. T2V records need text embeddings and video latents. I2V records
may additionally need first-frame latents and CLIP features. A merged dataset
uses a videos/ directory and videos2caption.json; an HF dataset uses video
and caption columns. Read data formats and run
the bundled manifest validator before GPU work.
The preprocessing entry point accepts --mode preprocess, workload type,
dataset type (hf or merged), dataset paths/output, video loader, target
height/width/frame count/FPS, batching, and worker controls. Output is a
combined Parquet dataset containing serialized latent/embedding bytes and shape,
dtype, and sample metadata. Downloading data and encoding it are expensive;
start with a tiny validated sample.
New work uses fastvideo.train: YAML config, Trainer = Method × Model × Callbacks × Config, and python -m fastvideo.train.entrypoint.train --config CONFIG [--dry-run]. Methods own losses/optimization, models own forward and
trainable parameters, callbacks own lifecycle hooks. Existing shipped recipes
may use the independent legacy fastvideo.training stack; never cross-import
those stacks. Read training workflow and
training troubleshooting.