---

name: rl-post-training description: "Run and troubleshoot RD-Agent RL post-training and AutoRL-Bench experiments with explicit model, benchmark, workspace, grading, and backend evidence." metadata: disco-role: operating parent-skill: rd-agent license: MIT disable-model-invocation: true

RD-Agent RL post-training

Use this sub-skill for rdagent RL post-training, autorl_bench, agent-driven GRPO/RL experiments, benchmark grading servers, and result collection.

Pick the surface

RL post-training loop

The application entry point accepts a base model, benchmark, step/loop counts, and an overall timeout. The implementation can evaluate a baseline during scenario initialization and then run iterative training. Make the model/data directories and backend explicit before launching:

python rdagent/app/rl/loop.py \
  --base-model <model-name> \
  --benchmark <benchmark> \
  --loop-n 1 \
  --timeout <bounded-duration>

Use the active CLI help or module source to confirm optional flags. The scenario may need GPU, Docker/conda environments, model weights, and benchmark data.

AutoRL-Bench

Probe without running a benchmark:

python -m rdagent.scenarios.rl.autorl_bench.run --help

A real run requires an agent id, task, base model, and timeout. It may download model/data assets and starts a local grading server:

python -m rdagent.scenarios.rl.autorl_bench.run \
  --agent example_agent \
  --task gsm8k \
  --model <huggingface-model-id> \
  --timeout <seconds> \
  --port <free-local-port>

Use example_agent and a tiny supported task only as a pipeline smoke test when the required assets are already available; do not call a long run a smoke test if it downloads gigabytes or trains a model.

Workspace and result contract

Each AutoRL-Bench run gets an isolated workspace under the scenario workspace root. The runner prepares model/data links, starts a grading server, records baseline and submissions, writes run logs and scores, and appends a summary row to results.csv. Preserve the workspace path, run.log, agent.log, scores.json, baseline, best score, submission count, timeout, and success flag.

Important environment/configuration signals include AUTORL_FILE_PATH, AUTORL_RDAGENT_ROOT, SMITH_BENCH_DIR, and provider variables for the driver agent. The default resource root is a git-ignored git_ignore_folder/rl_files relative to the checkout; override it rather than storing weights in source control.

Backend and data boundaries

• Model execution and realistic training normally require CUDA, compatible PyTorch/TRL/vLLM/OpenCompass-style dependencies, and enough disk.
• Some benchmarks require extra packages or Java; install only the benchmark-specific requirements for the selected task.
• Missing SMITH_BENCH_DIR can produce an empty Smith registry warning. Confirm registry discovery before interpreting it as “no benchmark.”
• Interrupts should terminate the whole child process group and preserve the workspace; inspect logs before restarting on a new port.

Read benchmark-contract.md and the parent troubleshooting guide before a run.