---

name: training-evaluation description: "Use segmentation_models_pytorch losses, metrics, tensor modes, and safe training/evaluation loops for segmentation." disable-model-invocation: true metadata: disco-role: operating license: MIT

training-evaluation

Use this sub-skill when an agent needs to train or evaluate segmentation with segmentation_models_pytorch (SMP) losses and metrics, diagnose mask/logit shape errors, or adapt a short training loop without downloads or long-running training.

Use When

• Selecting DiceLoss, JaccardLoss, TverskyLoss, FocalLoss, LovaszLoss, SoftBCEWithLogitsLoss, SoftCrossEntropyLoss, or MCCLoss.
• Deciding between BINARY_MODE, MULTICLASS_MODE, and MULTILABEL_MODE for logits, masks, losses, and metrics.
• Calling smp.metrics.get_stats followed by iou_score, f1_score, accuracy, precision, or recall.
• Building a minimal PyTorch training/evaluation loop around an SMP model.
• Debugging shape, dtype, threshold, from_logits, ignore_index, or class_weights failures.

Route Elsewhere

• Choose architectures, classes, in_channels, decoder options, or smp.create_model in model-building.
• Choose encoder weights and input normalization in encoders-preprocessing.
• Save, trace, script, ONNX-export, or package trained models in model-export.

Core Contract

• Binary segmentation: model output is usually (N, 1, H, W) and target masks are 0/1, preferably (N, 1, H, W) for metrics and either (N, H, W) or (N, 1, H, W) for most losses.
• Multiclass segmentation: model output is (N, C, H, W), target is integer class ids (N, H, W), and classes are mutually exclusive.
• Multilabel segmentation: model output and target are (N, C, H, W), each channel is an independent 0/1 label, and classes may overlap.
• Losses usually consume logits with from_logits=True; metrics consume hard integer masks or floating binary/multilabel scores plus a threshold.

References

• Losses and metrics covers modes, signatures, tensor shapes, reductions, and metric thresholds.
• Training workflows gives compact binary, multiclass, and multilabel loop skeletons.
• Troubleshooting maps common SMP errors to fixes.
• validate_training_shapes.py validates tiny deterministic shape/mode/loss/metric combinations and prints JSON.

Safe Smoke Check

Run the bundled validator before changing a real loop:

python sub-skills/training-evaluation/scripts/validate_training_shapes.py --mode binary --batch 2 --classes 1 --height 8 --width 8 --from-logits --metric-threshold 0.5
python sub-skills/training-evaluation/scripts/validate_training_shapes.py --mode multiclass --classes 3 --loss soft-ce
python sub-skills/training-evaluation/scripts/validate_training_shapes.py --mode multilabel --classes 4 --metric-threshold 0.5

The helper constructs tiny tensors only; it does not download data, build a model, or train.