---

name: data-preparation description: "Route for SSD Keras dataset parsing, validation, augmentation, and batch generation." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

Data Preparation

Use this route when the task is about getting object-detection data into a shape that the SSD workflows can use safely.

What this route covers

• parsing CSV annotations
• parsing Pascal VOC XML annotations
• parsing COCO-style JSON annotations
• loading and reloading HDF5 datasets
• validating boxes, labels, and image sizes
• generating batches with DataGenerator
• applying resize and augmentation chains
• converting predictions back to the original image frame with inverse transforms

What this route excludes

• building or compiling models
• loss configuration
• decoding predictions or evaluating mAP
• transfer learning / weight sampling

Send those tasks to training or inference-evaluation.

First things to check

1. Open references/data-formats.md to confirm the expected annotation layout.
2. Open references/compatibility.md if imports are failing in the environment.
3. Open references/model-architecture.md if the data must feed SSDInputEncoder.
4. Run scripts/smoke.py when you want a tiny synthetic parser / generator check.

Typical workflow

1. Choose the source format

• CSV is easiest for ad hoc detection datasets.
• VOC XML is the best fit for the Pascal VOC splits used in the notebooks.
• JSON is the right path for COCO-style datasets and COCO evaluation.
• HDF5 is useful when you want a reusable cached dataset for faster iteration.

2. Verify label conventions

• Keep background as class 0 on the model side.
• Make sure the label order is the one your parser and generator expect.
• Confirm that every transform sees the same labels_format mapping.

3. Parse and validate

• Use the parser that matches the dataset source.
• Drop or flag degenerate boxes early.
• Decide whether empty images should stay in the batch.
• If boxes disappear after cropping or scaling, loosen the validity filters before changing the labels themselves.

4. Generate batches

• Use Resize plus ConvertTo3Channels for a simple deterministic smoke path.
• Add augmentation chains only after the basic parser / generator flow is stable.
• For training, wire the generator to SSDInputEncoder through the training route.
• For evaluation, keep the inverse transforms so predictions can be mapped back to the original image coordinates.

5. Cache when it helps

• If the source annotations are expensive to parse repeatedly, build an HDF5 cache.
• Reuse the cache only when the source images and labels are still the same.

Useful source objects

• DataGenerator
• parse_csv
• parse_xml
• parse_json
• create_hdf5_dataset
• generate
• BoxFilter
• ImageValidator
• BoundGenerator
• Resize
• ConvertTo3Channels
• SSDDataAugmentation
• DataAugmentationConstantInputSize
• DataAugmentationVariableInputSize
• DataAugmentationSatellite
• apply_inverse_transforms

Quick decision guide

• Need a safe parser smoke? Use the bundled script and a tiny synthetic fixture.
• Need the full training path? Go to training after the data contract is fixed.
• Need evaluation or visualisation with transformed images? Keep the inverse transforms and go to inference-evaluation.