---

name: data-preparation description: "Prepares LTX training datasets by validating manifests, drafting scene-splitting and preprocessing commands, and checking cached latents, captions, references, and masks before training." disable-model-invocation: true metadata: disco-role: operating license: NOASSERTION

Data Preparation

Use this sub-skill when the request is about getting LTX training data into a safe, trainable shape: manifest validation, scene splitting, caption planning, preprocessing command drafting, reference or mask prep, and inspection of cached .precomputed outputs.

Route away

• Training mode selection, config authoring, launch, resume, or monitoring → ../training-workflows/SKILL.md
• Inference or validation with trained LoRAs → ../inference-pipelines/SKILL.md
• GPU / VRAM / OOM / backend tuning → ../performance-backends/SKILL.md

Fast paths

• Validate or normalize a manifest: scripts/validate_dataset_manifest.py
• Draft a preprocessing command: scripts/build_preprocess_command.py
• Run the bundled preprocessing launcher after approval: scripts/process_dataset.py
• Draft a scene-splitting command: scripts/build_scene_split_command.py
• Run the bundled scene splitter after approval: scripts/split_scenes.py
• Inspect precomputed latents and masks: scripts/inspect_precomputed_latents.py
• Use the bundled preprocessing internals only when you need to inspect or reuse them directly: scripts/process_videos.py, scripts/process_captions.py, and scripts/decode_latents.py

What this sub-skill covers

• CSV, JSON, and JSONL manifests with caption, media, reference, and mask columns.
• Resolution buckets and audio-duration rules for video, image, mixed, and audio-only datasets.
• Unified versus split checkpoint preprocessing flags.
• Captioning backend choice, credential boundaries, and spot-check approval.
• Reference and mask preprocessing layouts for IC-LoRA and inpainting.
• Safe checks for stale or mismatched cached outputs.

Working rules

• Do not start captioner servers, split videos, encode media, or download models here.
• Use the bundled scripts to validate inputs and draft commands; they are safe to run from any current directory.
• Prefer a fresh .precomputed/ directory when the model, Gemma root, trigger token, bucket list, or reference scale factors change.
• For mixed image + video datasets, plan multiple buckets that include F=1; the later training step must use batch size 1.
• For IC-LoRA, inpainting, and audio-only datasets, require the corresponding reference or audio inputs before preprocessing.

Read next

• references/dataset-formats.md
• references/captioning-and-preprocessing.md
• references/reference-and-mask-prep.md
• references/troubleshooting.md