---

name: training-data description: "Plan and operate iGAN dataset preparation, HDF5 schema checks, DCGAN training, batchnorm, predictor, packing, and training configuration workflows." disable-model-invocation: true metadata: disco-role: operating license: MIT

iGAN Training Data Router

Use this sub-skill when the task is about building or auditing the data and training side of iGAN rather than loading a finished model for inference. It is intentionally safe to load in a modern environment: the bundled scripts only plan, inspect, and print commands; they never download datasets, write HDF5 files, start Theano, train networks, or touch a GPU.

Route Here

• Plan which public HDF5 dataset archive is needed and how much disk/network budget it implies.
• Preflight a custom image directory before converting it to iGAN's HDF5 layout.
• Explain the imgs dataset shape, channel convention, Fuel split metadata, and train/test slicing expected by the legacy loaders.
• Build a DCGAN training command sequence from an existing HDF5 file.
• Estimate the required post-training batchnorm steps for the generator, discriminator, and optional predictor.
• Plan predictor training for image-to-latent x -> z support.
• Pack trained cache files into a compact .dcgan_theano model.
• Upgrade an older packed model layout to the newer key layout.
• Add or compare entries in the training configuration table.
• Explain cache directories, sample grids, logs, model checkpoints, and web pages emitted during training.
• Advise how to extend iGAN with a custom Theano generative model class after training data and model packing are understood.

Route Elsewhere

• For loading a pre-trained .dcgan_theano model, sampling images, or model download planning, use model-inference.
• For interactive drawing UI behavior, PyQt4 windows, brush tools, and display issues, use interactive-ui.
• For non-UI constrained generation from color, mask, and edge inputs, use constraint-generation.
• For running image projection or reconstruction at inference time, use image-projection; keep predictor training and predictor batchnorm planning here.
• For general installation/import failures not tied to training scripts, check the root troubleshooting reference if one is available, then return here for training-specific symptoms.

First Checks

1. Identify the intended model name, image size, and channel count.
2. Decide whether the user already has an HDF5 dataset or only an image folder.
3. Confirm whether the work is planning-only or a real legacy training run.
4. For any real training run, require an explicit legacy Python/Theano/CUDA plan; modern CPU-only Python environments are enough for this sub-skill's helpers but not for native DCGAN training.
5. Check disk budget before suggesting a public HDF5 archive; several archives are multi-gigabyte compressed files.
6. Check cache and output paths before packing; packing only includes model files that already exist.

Safe Bundled Helpers

• Use scripts/igan_dataset_urls.py --list to list known public HDF5 archives, compressed sizes, and URL/target path plans without network access.
• Use scripts/igan_dataset_urls.py --dataset shoes_64 --output-dir datasets to reproduce the download target naming convention in dry-run form.
• Use scripts/inspect_dataset_plan.py --mode dir --dataset-dir images --width 64 --channel 3 --hdf5-file datasets/custom.hdf5 to preflight a custom image directory and report the intended imgs shape and train/test split without writing HDF5.
• Add --model-name hed_shoes_64 or another known config to the preflight script when you want width/channel mismatch warnings.
• Add --json to either helper when another tool needs deterministic structured output.

Main References

• training-workflows.md gives runnable command templates for dataset acquisition planning, HDF5 conversion, DCGAN training, batchnorm estimation, predictor training, packing, upgrading, and custom model extension.
• data-formats.md explains public dataset names, HDF5 schema, image preprocessing, Fuel split metadata, cache outputs, and packed model keys.
• configuration.md summarizes model configuration functions, default hyperparameters, CLI options, and path conventions.
• troubleshooting.md maps concrete symptoms to likely causes and recovery steps for legacy Theano/CUDA, Fuel/HDF5, OpenCV, predictor, packing, and script-typo failures.

Dataset Decision Pattern

1. If the user wants one of the public domains, run the URL planner first.
2. If network or disk budget is not approved, stop at a plan and mark the native dataset download case as skipped.
3. If the user has local images, run the dataset preflight helper before any conversion command.
4. Keep the conversion output name explicit; the training scripts otherwise assume datasets/<model_name>.hdf5.
5. Make the model configuration match the dataset width and channel count.

Training Decision Pattern

1. Start from an HDF5 file with an imgs dataset and train/test Fuel split.
2. Run DCGAN training only in a compatible legacy GPU environment.
3. Estimate DCGAN batchnorm after generator/discriminator checkpoints exist.
4. Train the optional predictor only when image projection through cnn or cnn_opt needs a learned x -> z network.
5. Estimate predictor batchnorm after predictor parameters exist.
6. Pack all available model and batchnorm artifacts into one .dcgan_theano file for downstream inference and UI workflows.

Corrected End-to-End Sequence

Use this corrected order when reconstructing the legacy shell recipe:

1. train_dcgan.py
2. batchnorm_dcgan.py
3. train_predict_z.py when predictor support is wanted
4. batchnorm_predict_z.py when predictor support is wanted
5. pack_model.py

The historical shell recipe misspelled the fourth command as batchnorm_precit_z.py; that file is not part of the workflow. Always use the correct batchnorm_predict_z.py command shown in the reference.

Verification Expectations

• The bundled helpers should run with python <script> --help on any modern Python 3 environment.
• The URL helper can be checked without network by comparing names, URLs, sizes, and target paths.
• The dataset preflight helper can be checked with a tiny directory of fake or real image-named files because it only inspects names and plans shapes.
• Native HDF5 creation is optional CPU verification if OpenCV, h5py, and Fuel are installed.
• Native DCGAN training, DCGAN batchnorm, predictor training, predictor batchnorm, and real projection-dependent checks are optional expensive CUDA cases unless a compatible legacy stack and artifacts are provided.

Operating Reminders

• Do not present planning helper output as proof that a model was trained.
• Do not hide legacy constraints: the original code targets Python2-era Theano, CUDA, cuDNN, Fuel, Lasagne, and OpenCV APIs.
• Do not download public archives or delete ZIP files from inside this skill; only show a plan and ask for explicit execution outside the helper if needed.
• Do not point users back to source checkout files for routine workflows; this sub-skill and its references contain the needed commands and contracts.
• Keep inference-only questions out of this sub-skill even when they mention a trained model file; route them to model inference unless the question is about creating, packing, or upgrading that file.