---

name: data-preparation description: "Prepare and validate KAIR-compatible image, video, LMDB, and meta-info datasets without mutating source data." metadata: disco-role: operating disable-model-invocation: true license: MIT

KAIR data preparation

Use this sub-skill when the task is to prepare, inspect, or explain KAIR dataset inputs before training or testing. It covers image folders, paired LQ/GT data, video clip folders, Vimeo90K sequence folders, meta-info files, subimage plans, and LMDB plans.

Do not use this sub-skill for model command ownership. Route image training to ../image-training/SKILL.md, image inference to ../image-testing/SKILL.md, and VRT/RVRT model commands to ../video-restoration/SKILL.md; return here only for dataset layout, meta-info, or conversion questions.

Safe operating rules

1. Treat KAIR as a source-script checkout: dataset tools in KAIR can be hard-coded and can write, move, copy, or delete data.
2. Prefer the bundled read-only checkers and planners in this sub-skill before any conversion or regrouping.
3. Before recommending a writer, state the expected input tree, output tree, key convention, and whether the operation is destructive or expensive.
4. Never tell the user to run a hard-coded preparation script blindly. If they still want to use one, require a reviewed copy of the data and confirmed paths.
5. Testing datasets for VRT/RVRT normally use frame folders directly; LMDB is mainly for training datasets.

References

• references/data-layouts.md — train/test folder conventions, video layouts, meta-info formats, LMDB structure, and the dataset_type mapping.
• references/data-preparation-workflows.md — safe plans for DIV2K subimages/LMDBs, REDS, DVD, GoPro, UDM10, DAVIS, Vimeo90K, Set8, and MATLAB-only steps.
• references/troubleshooting.md — empty datasets, LQ/GT mismatches, existing output exits, destructive scripts, missing meta-info, LMDB conflicts, libpng fallback, and multiprocessing memory issues.

Bundled read-only scripts

Run these from any machine with Python 3; they do not import KAIR and do not write data.

python sub-skills/data-preparation/scripts/check_dataset_layout.py --help
python sub-skills/data-preparation/scripts/check_dataset_layout.py image --root trainsets/trainH
python sub-skills/data-preparation/scripts/check_dataset_layout.py video --root testsets/REDS4/sharp_bicubic --paired-root testsets/REDS4/GT
python sub-skills/data-preparation/scripts/check_dataset_layout.py lmdb --root trainsets/REDS/train_sharp_with_val.lmdb

python sub-skills/data-preparation/scripts/plan_lmdb_conversion.py --dataset div2k
python sub-skills/data-preparation/scripts/plan_lmdb_conversion.py --dataset reds

Quick routing

• "Create a DIV2K LMDB" → use references/data-preparation-workflows.md#div2k-subimages-and-lmdbs, then scripts/plan_lmdb_conversion.py --dataset div2k.
• "Check REDS layout for VRT" → use references/data-layouts.md#video-folder-layouts and scripts/check_dataset_layout.py video on both LQ and GT roots.
• "Why no dataset found?" → use references/troubleshooting.md#empty-dataset-or-no-images-found and check folder depth, extensions, and folder_lq/folder_gt pairing.
• "Prepare GoPro as video" → use references/data-preparation-workflows.md#gopro-video-deblurring-layout; warn that the original regrouping script moves folders and deletes the original train/test folders.
• "List KAIR dataset_type values" → use references/data-layouts.md#dataset_type-mapping.