---

name: data-preparation description: "Prepare, validate, and troubleshoot DeepMedic 0.8.4 NIFTI subjects, modality manifests, labels, ROI masks, CSV inputs, and intensity normalization before training, validation, or inference." disable-model-invocation: true metadata: disco-role: operating license: NOASSERTION

DeepMedic data preparation

Use this skill when a Researcher must turn a new multi-modal NIFTI dataset into DeepMedic input, or diagnose a path, list, grid, label, ROI, or normalization failure. Establish the subject ordering and spatial contract first; only then hand the prepared inputs to the session-specific skill.

Route by need

• Read references/data-formats.md for the canonical NIFTI, manifest, CSV, label, ROI, and normalization contracts.
• Read references/troubleshooting.md when a preflight check, loader warning, shape error, label error, or z-score result is unexpected.
• Run the safe standalone checker at scripts/validate_nifti_manifest.py for explicit channel list files. It does not import DeepMedic or mutate data.
• After data passes this skill, route architecture decisions to ../model-architecture/SKILL.md, training execution to ../training/SKILL.md, and testing/output handling to ../inference/SKILL.md.

Required preparation workflow

1. Define the subject table. Give every subject one row/order. For each row, record the modality paths in fixed channel order, and optionally one label and one ROI path. Never independently sort one modality list after assembling the table.
2. Choose one input representation. Use classic per-channel list files or the supported dataframe branch. For list files, put one data path per line; relative data paths are relative to that list file. For CSV, use channel_<name> columns and remember that the implementation sorts those column names alphabetically. See the data-formats reference for required ground_truth, optional roi_mask, and optional prediction_filename signals.
3. Prepare the spatial grid. All modalities, labels, and ROI masks for a subject must be co-registered and have the same voxel-array shape and voxel size. Resample/crop/orient the complete subject set together before making manifests. Use one voxel size throughout the database. Do not expect DeepMedic to register or resample files at load time.
4. Prepare labels and ROI. Training labels are required. Background is 0 and task classes are contiguous increasing integers (0, 1, 2, ...); with N output classes, values must be in [0, N). ROI is optional: absent ROI means whole-volume sampling/inference and whole-volume normalization statistics. A provided ROI uses positive voxels as inside.
5. Validate without changing data. Run the bundled checker once for all channel list files, with optional labels/ROI. It checks list lengths, file existence, NIFTI loading/dimensionality, per-subject shape and voxel-size agreement, and optional affine and label constraints. Use --read-data for a full payload read. Also inspect voxel-size consistency across subjects and verify co-registration visually or with a trusted imaging QA process; those are not fully proven by a header check.
6. Decide normalization explicitly. Prefer data whose intensity policy is known. If applying runtime z-score, configure one mode only: all channels or a boolean per-channel list. Statistics use positive ROI voxels, or the full volume without ROI; cutoffs only control statistic estimation. Check for an empty/constant selected region and avoid double-normalizing exported data.
7. Record the handoff. Preserve the final channel order, case count, input representation, grid/resampling policy, label set/class count, ROI policy, normalization parameters, validator result, and any intentional missing modality (-) in the task notes. A later session skill should consume this record rather than reconstructing it from filenames.

Exact runtime contracts

The source-backed data I/O and preprocessing signatures are:

load_volume(filepath)

saveImgToNiiWithOriginalHdr(imgToSave, filepathTarget,
                            filepathOriginToCopyHeader,
                            npDtype=np.dtype(np.float32), log=None)

savePredImgToNiiWithOriginalHdr(labelImageCreatedByPredictions,
                                namesForSavingPreds,
                                listOfFilepathsToEachChannelOfEachPatient,
                                case_i, suffixToAdd="",
                                npDtype=np.dtype(np.float32), log=None)

saveFmImgToNiiWithOriginalHdr(fmImageCreatedByVisualisation,
                              namesForSavingPreds,
                              listOfFilepathsToEachChannelOfEachPatient,
                              image_i, index_of_typeOfPathway_to_visualize,
                              index_of_layer_in_pathway_to_visualize,
                              index_of_FM_in_pathway_to_visualize, log=None)

save4DImgWithAllFmsToNiiWithOriginalHdr(multidimImageWithAllVisualisedFms,
                                        namesForSavingFms,
                                        listOfFilepathsToEachChannelOfEachPatient,
                                        image_i, log=None)

normalize_zscore_subj(log, channels, roi_mask, prms,
                      verbose_lvl=0, job_id='', in_place=True)

parse_filelist(filelist_path, make_abs=False)
parse_fpaths_of_channs_from_filelists(list_of_filelists, abs_path_root)
get_paths_from_df(log, df, abs_path, req_gt=True)

load_volume returns a 3-D NumPy array: 2-D becomes x,y,1; 4-D is allowed only with singleton fourth dimension. The NIFTI save helpers copy the source header affine and relevant zooms; the generic helper creates parent folders and appends .nii.gz when the target does not already end in that suffix. Prediction/feature helpers derive output names from per-case name tokens and use the first channel as the header source. They are output helpers, not a replacement for input registration.

normalize_zscore_subj expects channels shaped [channels, x, y, z] and an ROI shaped [x, y, z] or None. prms contains apply_to_all_channels, apply_per_channel, cutoff_percents, cutoff_times_std, and cutoff_below_mean; incompatible all/per-channel selection is rejected. It returns (channels, applied), mutating the input by default. The long-form behavior and failure cases are in the references.

Scope boundary

This sub-skill does not choose CNN pathways, class-head architecture, patch sizes, optimizer settings, training schedules, checkpoint loading, or feature map inference. Do not “repair” a data error by changing those settings. Use the three linked sibling skills only after this input contract is satisfied.

Evidence and version caveat

This operating contract follows DeepMedic 0.8.4 source behavior for I/O, manifest parsing, dataframe extraction, subject loading, label checks, and normalization. The prose documentation calls dataframe input new/not yet used, while the 0.8.4 parser contains active dataframe branches; follow the code when the two conflict. The prose also says .nii, while the bundled examples use .nii.gz; the loader delegates to NiBabel and the examples show that both common NIFTI suffixes are an intended safe contract. The bundled example names illustrate two modalities, labels, and brain masks only; they are not a requirement that a new task have exactly two channels or a brain ROI.