---

name: detection description: "Use for PaddleViT object detection workflows with DETR, Swin, or PVTv2: validate COCO data, select configs, build/train/evaluate models, reason about transforms, losses, post-processing, and run safe utility smokes. Excludes segmentation and generic export; cross-link deployment-and-operations for shared runtime concerns." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

PaddleViT object detection

Use this route for standalone PaddleViT object detection under object_detection/: DETR, Swin, and PVTv2 with COCO boxes. It covers family/config selection, data preflight, transforms and targets, safe model-contract checks, checkpoint triage, training/evaluation boundaries, and backend-aware diagnosis. It does not own semantic segmentation, generic export/inference, quantization, or weight porting; route those requests to deployment-and-operations.

The bundled helpers are self-contained and do not import the original source checkout, download data or weights, or modify a user-owned dataset. A source checkout is optional and is needed only for a source-model build or native candidate test. Details that are useful during a focused investigation live in model overview, data formats, workflows, and troubleshooting.

Route the request

1. Identify the family before changing a config or launching a command:
   • DETR for query-based, end-to-end detection with transformer decoder queries, Hungarian matching, and direct box post-processing.
   • Swin or PVTv2 for hierarchical backbones feeding FPN, RPN, and RoI-style heads with anchors, proposals, and NMS.
2. If the request says only “PaddleViT detection,” ask which family and whether the goal is a utility check, source build, evaluation, or training. Do not mix family directories in one Python process: common modules such as config, coco, box_ops, and utils can resolve incorrectly.
3. Keep segmentation separate, even if a request mentions masks or a DETR import. For export, predictor, AMP, distributed, or shared runtime issues, use deployment-and-operations.

Preflight before a source run

Validate the COCO root, not a split directory. The expected root contains annotations/instances_{train,val}2017.json and the matching train2017/ or val2017/ image directory. Run the read-only helper from the skill root (replace placeholders with caller-owned values):

python <skill-root>/scripts/check_coco_layout.py <coco-root> --split val

Add --check-images to decode referenced images and compare dimensions, or --check-api to parse the annotation through pycocotools. Use --json for a machine-readable report and --demo for a temporary one-image validator fixture. A failed check is a stop condition: repair the dataset outside this skill and rerun; do not download or silently substitute another dataset.

Before importing a source model, run the bounded synthetic contract smoke:

python <skill-root>/scripts/detection_model_smoke.py --model all --device cpu

This checks tiny DETR and four-level anchor-family shapes and finite values. It is not a source-model build, checkpoint test, COCO test, mAP result, or benchmark reproduction. Use --device gpu:0 only when a GPU claim is required and the backend has already passed its environment probe.

Select a source root and config

The three projects are standalone. For a source-backed run, enter exactly one family directory and expose only that directory (and its intended parent) to imports:

cd <source-root>/object_detection/DETR   # or Swin or PVTv2
export PYTHONPATH="$PWD:$PWD/..:${PYTHONPATH:-}"

Resolve configuration in this order: Python defaults, recursive YAML BASE files, then CLI overrides. Print and inspect the effective config before model construction. Check backbone output channels against FPN inputs, pyramid strides and anchor levels, class/category conventions, image divisibility, and DETR embedding-dimension/head divisibility. Swin/PVTv2 configs preserve the historical ROI.NUM_ClASSES spelling; do not replace it with a guessed key. The family-specific config and output/target details are in model overview and data formats.

CLI flag contract

The family main_single_gpu.py and main_multi_gpu.py scripts use these short, single-dash options:

-cfg PATH             YAML config (BASE files are recursively merged)
-dataset coco         dataset selector
-data_path PATH       COCO root, not train2017/val2017
-batch_size N         per-process/per-GPU batch size
-eval                 evaluation-only mode
-pretrained PREFIX    source appends .pdparams
-resume PREFIX        source expects .pdparams and .pdopt
-last_epoch N         resume epoch metadata
-ngpus N              configured multi-GPU worker count

-cfg=... and -cfg ... are both acceptable. Pass checkpoint prefixes without .pdparams when following the source loader. Inspect launcher shell files rather than executing them blindly: paths, visible devices, output prefixes, and training duration are caller-owned. A representative command shape is:

cd <source-root>/object_detection/DETR
CUDA_VISIBLE_DEVICES=0 python main_single_gpu.py \
  -cfg=./configs/detr_resnet50.yaml -dataset=coco \
  -batch_size=1 -data_path=<coco-root> -eval \
  -pretrained=<checkpoint-prefix>

Treat real training/evaluation as expensive and data/checkpoint/GPU dependent; use the workflow reference for gated sequencing.

Family contracts to preserve

• DETR: expect logits shaped [B,Q,C+1], normalized center-size boxes [B,Q,4], optional auxiliary decoder outputs, and losses including classification, L1, and GIoU. Post-processing needs target sizes in [height,width] order and emits absolute xyxy boxes, scores, and labels.
• Swin/PVTv2: expect hierarchical features, FPN/RPN/RoI losses during training, and post-NMS rows shaped [label, score, xmin, ymin, xmax, ymax] during evaluation. Their target path uses absolute boxes and contiguous classes; the two families share the broad head/neck contract but not every backbone channel/config value.
• All families: COCO boxes begin as [x,y,width,height]; transformations must update geometry and area. A valid synthetic training fixture needs at least one non-empty target. Preserve original image IDs for COCO results. Full transform and output schemas are in the linked references.

Verification tiers

Choose the cheapest tier that answers the question and record command, device, Paddle version, config, and result:

1. check_coco_layout.py for root layout, JSON arrays, IDs, boxes, files, and optional Pillow/COCO-API checks.
2. detection_model_smoke.py for standalone tiny shape and finite-value contracts; a non-32-divisible size intentionally reports the padding caveat.
3. Native DETR box/transform/model tests when a source checkout and dependencies are available. They are evidence candidates, not runtime prerequisites; some are skipped or require an untracked fixture.
4. A one-batch source build/forward using one family root and a tiny local fixture, only after config/import checks pass.
5. Real COCO evaluation or training only with explicit data, checkpoint, GPU, and time approval. Never call a smoke or random-weight forward mAP evidence.

CPU can validate parsing, layout, box utilities, and tiny diagnostics. It does not establish CUDA, AMP, distributed, or full detector claims. For those boundaries use deployment-and-operations.

Stop and recover

• Missing files/API: -data_path must be the root; verify exact split filenames. If pycocotools is absent, omit --check-api but do not claim COCO evaluation is verified.
• Import/YAML errors: start a fresh process, isolate one family root, resolve BASE relative to its YAML, compare keys to that family's config, and inspect the final config. Do not mix module paths or claim a build that never constructed a model.
• Shape/checkpoint errors: verify family, backbone/FPN channels, attention divisibility, query/class counts, head spelling, and checkpoint provenance. A prefix normally maps to .pdparams; resume also needs .pdopt. Do not reshape incompatible state dictionaries.
• Bad boxes/empty batches: inspect crowd flags, category mapping, positive area, coordinate format, resize/flip/crop order, and image IDs. Keep one valid target in a fixture; an all-empty batch is not a valid smoke.
• NaN/Inf or no detections: run CPU geometry checks, verify normalized DETR boxes or absolute Swin/PVTv2 boxes, then inspect thresholds, NMS, and loaded keys. Random weights cannot support an accuracy claim.
• CUDA OOM/unavailable: lower per-GPU batch size or input scale for OOM. If CUDA is unavailable, report CPU-only partial evidence; never substitute it for a required GPU result. For multi-GPU, check visible devices, -ngpus, per-process batch semantics, NCCL, and rank gathering before retrying.

See troubleshooting for expanded recovery branches. Do not run network, full-native, long-training, or multi-GPU workflows as part of this route unless explicitly authorized.