---

name: inference description: "Build and run NanoTrack V1/V2/V3 for stateful image or video inference with safe model, config, device, frame, box, and output handling." disable-model-invocation: true metadata: disco-role: operating license: NO_LICENSE

NanoTrack Inference

Use this sub-skill for headless, single-object NanoTrack inference on an image sequence or decoded video frames. It covers V1, V2, and V3 model construction, checkpoint loading, tracker state, CPU/CUDA placement, and safe adaptation of a demo loop.

Route First

• Dataset protocols, benchmark runners, result files, metrics, or hyperparameter search: route to evaluation.
• Training datasets, augmentation, losses, resume, or distributed launches: route to training.
• ONNX, NCNN, model splitting, latency, or deployment performance: route to export.
• SiamFC, SiamBAN, Ocean, or any snapshot other than the maintained NanoTrack workflow: route to variant-catalog.

Do not use this sub-skill to claim benchmark quality, training recovery, export correctness, or speed. Those require different assets and verification.

Read By Need

• Public classes, config fields, variant matrix, and input/output contracts: API reference
• Complete model setup and headless frame-loop patterns: workflows
• Failure diagnosis and safe recovery: troubleshooting
• Deterministic preflight/config generator: nanotrack_demo_check.py

Non-Negotiable Inference Order

1. Choose exactly one variant: v1, v2, or v3.
2. Start a fresh Python process when changing variants. NanoTrack uses one mutable module-global cfg; merges are not isolated.
3. Merge the matching YAML before constructing ModelBuilder().
4. Select the matching ban_v1, ban_v2, or ban_v3 head implementation. YAML alone does not select the head implementation.
5. Resolve one device, set cfg.CUDA from that choice, and move the model to the same device.
6. Construct ModelBuilder() with no arguments, load the matching checkpoint, call .to(device).eval(), then call build_tracker(model).
7. Validate a BGR numpy.ndarray frame and a zero-based [x, y, w, h] box.
8. Call tracker.init(first_frame, box) once, then call tracker.track(next_frame) in temporal order.
9. Consume result["bbox"] and result["best_score"]; validate them before drawing, serializing, or feeding another system.

Never copy a demo's unconditional .cuda(), implicit current-directory paths, GUI creation, webcam access, warm-up frame skipping, or output writer side effects into a library/server workflow.

Fast Preflight

The bundled checker does not import NanoTrack, download weights, decode video, or open a window. Its default is a deterministic CPU synthetic case:

python /path/to/inference/scripts/nanotrack_demo_check.py

Validate explicit inputs:

python /path/to/inference/scripts/nanotrack_demo_check.py \
  --variant v3 \
  --frame-shape 720 1280 3 \
  --bbox 410 180 120 96 \
  --device cpu \
  --config /path/to/config.yaml \
  --checkpoint /path/to/nanotrackv3.pth \
  --require-checkpoint --json

Generate a self-contained inference-only YAML rather than copying historical training configuration:

python /path/to/inference/scripts/nanotrack_demo_check.py \
  --variant v2 --write-config ./nanotrack-v2-inference.yaml

Generation is explicit and refuses to overwrite by default. It writes no checkpoint and performs no network access.

Variant Decision

• Choose V1 only with a V1 checkpoint and ban_v1 head.
• Choose V2 only with a V2 checkpoint and ban_v2 head.
• Choose V3 only with a V3 checkpoint, ban_v3 head, 96-channel backbone/neck/head configuration, and output size 15.
• V1 and V2 both use the 64-channel backbone path and output size 16, but their head code, checkpoints, and tracking hyperparameters are not interchangeable.
• All maintained variant YAMLs use point stride 16. The core default is stride 8 and is not the effective maintained variant setting.

See the exact values in the variant matrix.

Stateful Lifecycle

NanoTracker(model) owns position, target size, channel mean, Hanning window, and point-grid state. init also calls model.template(...), which stores the template feature on the model itself. Consequences:

• track before init is invalid.
• Reinitialize for a new target, seek, scene cut, or new video.
• Preserve chronological frame order; the returned box updates the next search.
• Two trackers that share one ModelBuilder are not independent because the most recent init overwrites the model's template feature. Use one model per concurrent target or explicitly serialize and restore templates.
• Do not mutate geometry fields after tracker construction; its point grid and window were already derived from config.

Frame And Box Gate

Accept frames only when all of these hold:

• numpy.ndarray, shape [height, width, 3], nonempty, finite dimensions;
• BGR channel order, normally contiguous uint8 from OpenCV;
• the same channel convention and compatible resolution across the sequence.

Accept initialization boxes only when they are four finite numbers in zero-based [x, y, width, height] form with positive width and height. Prefer a box fully inside the first frame. For a deliberately partial box, require a nonempty intersection and make clipping policy explicit before init.

The returned box is a four-element floating-point [x, y, w, h] list. Internal center/size clipping does not guarantee that every returned corner is inside the image, so clip only at the consumer boundary when required. best_score is the selected foreground softmax score; treat it as a confidence-like ranking signal, not as a calibrated probability or benchmark metric.

Device Contract

cfg.CUDA controls whether tracker crops call .cuda(). Model placement is separate. They must agree:

device = torch.device("cuda" if requested_cuda and torch.cuda.is_available() else "cpu")
cfg.CUDA = device.type == "cuda"
model = load_pretrain(ModelBuilder(), checkpoint_path)
model = model.to(device).eval()

If CUDA was explicitly required and unavailable, fail instead of silently changing the experiment. If CPU fallback is allowed, set cfg.CUDA = False before init; changing only the model device is insufficient.

Evidence Limits

No checkpoint or dataset is bundled. The checker proves argument, profile, config, frame-shape, box, and device-selection behavior only. Full tracking requires a compatible checkpoint and real decoded frames. A backend preparation smoke is not evidence of model/checkpoint compatibility or end-to-end accuracy.