---

name: policy-eval description: "Configure RoboTwin XPolicyLab policy evaluation, remote policy servers, scheduler dry-runs, and action-adapter checks." metadata: disco-role: operating disable-model-invocation: true license: MIT

RoboTwin policy evaluation

Use this sub-skill when the user asks to evaluate a policy in RoboTwin, configure XPolicyLab local or remote evaluation, debug scheduler configs, validate policy action formats, or interpret evaluation result layouts.

Route first

• For simulation/task-class/render setup, read simulation-core.
• For trajectory datasets used to train/evaluate policies, read data-pipeline.
• For adding new tasks or language templates, read task-authoring.

Prerequisites

• A RoboTwin workspace with assets installed and simulation dependencies working.
• If you only have the generated skill tree, use the root workspace bootstrapper first to materialize a pinned public workspace.
• The XPolicyLab submodule initialized and a policy adapter under XPolicyLab/policy/<policy_name>/.
• Separate policy and evaluation environments when the policy stack and RoboTwin simulation dependencies differ.
• GPU IDs and capacity chosen for the host.

Main workflows

• Single/local evaluation and multitask scheduling: read evaluation-cli.md. Use --dry-run first.
• Remote policy server + local simulator deployment: read remote-server.md.
• Observation/action adapter semantics: read adapter-contract.md and run scripts/check_action_adapter.py for synthetic qpos/endpose shape checks.
• Config rendering without launching jobs: use scripts/render_eval_config.py to inspect a scheduler YAML's effective task list and GPU slots.
• Failure diagnosis: read troubleshooting.md.

Evaluation modes

1. bash scripts/eval_policy.sh serve --config env_cfg/eval/remote_server.yml starts a pool of policy servers.
2. bash scripts/eval_policy.sh multitask --config env_cfg/eval/all_tasks.yml ... schedules local policy+simulator jobs or local simulator clients against remote servers.
3. Without multitask or serve, scripts/eval_policy.sh dispatches a single eval call to the XPolicyLab adapter path.

Validation signals

• --dry-run prints the schedule and concrete commands without launching policies.
• Scheduler config rejects unknown fields, duplicate tasks, invalid GPU IDs, missing checkpoint in local mode, and missing remote endpoints in remote mode.
• eval_result/multitask/<run_id>/summary.json reports jobs total/succeeded/failed/skipped.
• Per-task logs are under eval_result/multitask/<run_id>/logs/.
• Single task eval writes _result.txt under eval_result/<task>/<policy>/<task_config>/<checkpoint>/<timestamp>/.