---

name: evaluation description: "Evaluate NAVSIM trajectories with cached metrics, one-stage or two-stage EPDMS, configured traffic policies, and reproducible CSV outputs." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

NAVSIM evaluation

Use this route when a trajectory or agent must be scored with NAVSIM v2 EPDMS, when metric caches need to be prepared, or when a CSV result needs diagnosis. This route covers the simulation/scoring contract, not model design or training. For agent output and sampling contracts, use the sibling agents route; for workspace and split path setup, use setup-and-data.

Route

1. Read workflows and run the read-only evaluation-config inspector first: python scripts/inspect_evaluation_config.py --help.
2. Select the exact split and stage. Make the metric-cache split, log/synthetic scene roots, and evaluation split agree before any data-backed command.
3. Follow configuration to choose a worker, agent/submission input, proposal sampling, scorer, output directory, and traffic policy. Use the command templates; replace placeholders explicitly.
4. If a cache is absent or stale, run the cache command only after checking maps, logs, split filters, and storage. Cache and evaluation must use the same proposal sampling and scene universe.
5. Run one-stage or two-stage scoring as described in workflows. Treat warnings, failed-token counts, and pseudo-closed-loop validity as acceptance gates rather than cosmetic log messages.
6. Inspect the CSV using metrics-reference and stop if the expected summary row, stage columns, or valid-scenario status is absent. For failures, use troubleshooting.

Do not download data, submit results, or launch a benchmark workload by default. The bundled inspector is diagnostic only and never opens datasets or caches.