---

name: data-loading description: "Load, inspect, split, decode, benchmark, and troubleshoot existing TensorFlow Datasets datasets." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Data Loading

Use this sub-skill when the task is to consume an existing TensorFlow Datasets (TFDS) dataset: inspect metadata, select splits, load tf.data.Dataset objects, create NumPy/Python data sources, customize decoding, visualize examples, benchmark iteration, or troubleshoot read/download failures.

Do not use this sub-skill to author new dataset builders, run TFDS CLI workflows, configure Beam/Dataflow generation, or design external/community/folder dataset formats. Route those tasks to the sibling sub-skills dataset-authoring, cli-workflows, beam-and-performance, and formats-and-community respectively.

Fast operating path

1. For safe metadata-first exploration, run the bundled inspector without download:

   python scripts/tfds_inspect_dataset.py mnist --split train
   

   Add --download only when the user explicitly accepts dataset download/preparation cost.

2. Pick the loading surface:

   • tfds.builder(...) for metadata, configs, versions, prepared-state checks, and explicit download_and_prepare() / as_dataset() control.
   • tfds.load(...) for the standard tf.data.Dataset path.
   • tfds.data_source(...) for TensorFlow-less Python sequence / NumPy-style consumption after data exists in a random-access format.

3. Choose splits and determinism controls before iteration. Use split strings or tfds.even_splits(...), then configure shuffle_files, tfds.ReadConfig, and framework-level sharding deliberately.

4. Add decoding/performance changes only after confirming feature structure from builder.info.features or info.features returned by with_info=True.

5. If an error mentions missing manual files, checksum mismatch, GCS/authentication, unsupported visualization, random-access file format, or TensorFlow/Pandas/Torch/JAX boundaries, use the troubleshooting reference before retrying with downloads or broad optional dependencies.

Bundled references

• Loading workflows: recipes for tfds.load, tfds.builder, tfds.data_source, NumPy/DataFrame conversion, visualization, Keras, JAX, PyTorch, and GCS decisions.
• Splits, determinism, and performance: split string syntax, tfds.even_splits, ReadConfig, deterministic shuffling, multi-worker reads, tfds.benchmark, auto-cache, decoding, and memory controls.
• API reference: verified public signatures and option semantics for the loading APIs covered by this sub-skill.
• Troubleshooting: failure-mode table for no-download inspection, missing prepared data, manual downloads, checksum errors, GCS, optional dependencies, feature decoding, and framework boundaries.

Bundled script

• Dataset inspector: safely prints package version, builder metadata, splits, features, configs, and optional sample/spec information. It avoids downloads unless --download is passed.

Safety defaults

• Prefer download=False or tfds.builder(...) inspection when the user only asks to inspect or debug metadata.
• Treat download=True, builder.download_and_prepare(), public GCS streaming, and manual download paths as potentially network-, storage-, or credential-sensitive.
• Do not promise TensorFlow-less usage from tfds.load; use tfds.data_source for Python sequence/DataLoader workflows and document that random-access formats such as ArrayRecord are expected.
• Keep deterministic and high-throughput training separate: shuffle_files=True improves shard-level shuffling for training but changes read determinism unless controlled with ReadConfig.