---

name: data-config description: "Inspect DeePMD-kit data systems and draft or repair training input configuration before training or testing." disable-model-invocation: true metadata: disco-role: operating license: LGPL 3.0

DeePMD-kit Data and Training Configuration

Use this sub-skill when the task is to prepare, inspect, validate, or repair DeePMD-kit system data directories and training input JSON/YAML before launching training.

Route Here When

• The user has a DeePMD system directory and asks whether it is valid.
• The task mentions type.raw, type_map.raw, nopbc, set.*, coord.npy, box.npy, labels, or mixed-type data.
• The task is to draft or repair training_data, validation_data, type_map, model, loss, learning_rate, or training sections.
• The user needs data/layout guidance for energy-force-virial, Hessian, tensor, DOS, property, spin, fparam, or aparam supervision.
• The task asks how to use dp doc-train-input, DP-GUI-style JSON schema output, HDF5 paths, LMDB systems, or dpdata conversion before training.

Route Elsewhere

• For choosing descriptors, fitting nets, loss schedules, launching dp train, freezing checkpoints, or reading training logs, use ../training-models/SKILL.md.
• For testing frozen models, DeepPot inference, dp test, model deviation, descriptors, compression, conversion, or bias changes, use ../inference-model-ops/SKILL.md.
• For installing DeePMD-kit, selecting TensorFlow/PyTorch/JAX/Paddle backends, or resolving package import problems, use ../installation-backends/SKILL.md.

Core Concepts

• A DeePMD system is one data folder or one supported data store entry; a frame is one configuration within a system.
• Standard NumPy systems group frames with the same atom count and atom-type ordering under one folder.
• type.raw is required and defines integer atom type indexes for the system; type_map.raw is optional for standard data but strongly recommended when model.type_map is set.
• nopbc marks a non-periodic system. Periodic systems need box.npy in each set.*; non-periodic systems should omit or ignore boxes consistently.
• Input frame properties include coord, box, fparam, aparam, and numb_copy/prob-style frame weighting.
• Label files are loaded only when the chosen model/loss/test path requests them; missing labels are harmless only when their loss prefactors or model features do not require them.
• Mixed-type systems use placeholder type.raw, required type_map.raw, and per-frame real_atom_types.npy; type -1 is a virtual atom used for padding.

First Inspection Pass

1. Identify the system path type: NumPy directory, HDF5 path using file.hdf5#/group, LMDB file, or raw text waiting for conversion.

2. For NumPy directories, run the bundled inspector:

   python sub-skills/data-config/scripts/inspect_deepmd_system.py /path/to/system --pretty
   

3. Read references/data-formats.md to interpret the summary and warnings.

4. Confirm that model.type_map matches the intended element order and is compatible with every system's type.raw or type_map.raw.

5. Confirm labels match the requested training target: do not add loss prefactors for missing labels.

6. For JSON/YAML input files, generate authoritative schema/reference output from the installed CLI when available:

   dp doc-train-input --out-type json_schema > deepmd-train.schema.json
   dp doc-train-input --out-type rst > deepmd-train-input.rst
   

Data Layout Workflow

• Use references/data-formats.md for the file tree, shape expectations, label ownership, HDF5/LMDB/raw conversion notes, and mixed-type rules.
• Use scripts/inspect_deepmd_system.py for safe local checks that do not import deepmd or mutate data.
• Treat the inspector as a preflight, not as a full DeePMD loader replacement: it flags likely shape and consistency errors, but the final backend loader may enforce additional descriptor/loss-specific constraints.
• For raw text data, convert to NumPy format before training. Raw coord.raw, box.raw, energy.raw, force.raw, and similar files are evidence/inputs for conversion, not a directly trainable format.
• For dpdata workflows, use dpdata to convert external molecular formats into DeePMD NumPy/HDF5 systems, then inspect the converted result here.

Training Input Workflow

• Use references/training-config-reference.md for the top-level structure, single-task and multi-task data sections, batch-size styles, and label-to-loss mapping.
• Start from a minimal config with top-level model, learning_rate, loss, and training for single-task workflows.
• Put systems under training.training_data.systems; use training.validation_data.systems only when validation data exists.
• For multi-task workflows, put per-task training_data, validation_data, and optional stat_file under training.data_dict.<task> and pair them with matching model.model_dict and loss_dict keys.
• Prefer explicit batch_size during debugging. Use auto, auto:N, max:N, or filter:N after confirming atom counts and memory expectations.
• Avoid expensive neighbor-stat surprises: descriptor selections such as sel: "auto" trigger neighbor statistics unless the training route explicitly skips or constrains it.

Label Ownership Quick Map

• Energy models: energy.npy, force.npy, optional virial.npy, hessian.npy, atom_ener.npy, atom_pref.npy, fparam.npy, aparam.npy.
• Hessian workflows: add hessian.npy and nonzero start_pref_h/limit_pref_h only for branches that own Hessian labels.
• Tensor workflows: use dipole.npy/atomic_dipole.npy or polarizability.npy/atomic_polarizability.npy according to global vs atomic tensor loss.
• DOS workflows: use dos.npy and optionally atom_dos.npy; the output dimension must match the fitting-net DOS grid.
• Property workflows: use the file named by the property fitting configuration, for example band_prop.npy.
• Spin workflows: include spin.npy when the model has spin inputs and force_mag.npy when magnetic force metrics/loss are required.
• Mixed-type workflows: include real_atom_types.npy in every set.*; virtual atoms use -1 and their atomic labels should be zero padded.

Repair Heuristics

• If atom labels load but forces are permuted or the model reports unexpected element errors, inspect type_map.raw versus model.type_map before changing model architecture.
• If periodic training fails on boxes, either add valid box.npy for every frame or add nopbc and remove PBC assumptions from downstream tasks.
• If validation finds missing virials, set virial loss prefactors to zero or provide virial.npy; do not leave nonzero virial weights with absent labels.
• If a folder contains multiple formulas, split by formula for standard data or convert to mixed-type with consistent Natoms/virtual atom padding.
• If JSON/YAML fails validation, regenerate schema with dp doc-train-input from the installed package and compare field names, aliases, and backend-specific options.
• If data loading is slow or neighbor statistics are expensive, reduce system discovery breadth, use explicit systems lists, set explicit neighbor selections, or debug with smaller converted systems.

References

• references/data-formats.md: self-contained data-system format, shape, label, mixed-type, HDF5, LMDB, and conversion guidance.
• references/training-config-reference.md: training input structure, data sections, batch options, schema generation, and label/loss alignment.
• references/troubleshooting.md: symptom-driven fixes for type maps, boxes, mixed-type, labels, invalid configs, sparse formulas, and neighbor statistics.
• scripts/inspect_deepmd_system.py: standalone NumPy-directory preflight inspector that prints JSON summary and warnings.