---

name: cleanrl description: "Use CleanRL's single-file reinforcement learning scripts, evaluation utilities, experiment operations, and repository maintenance workflows safely." disable-model-invocation: true metadata: disco-role: operating license: NOASSERTION

CleanRL Repo Skill

Use this skill when the user asks about CleanRL, its single-file RL algorithms, training commands, model evaluation/sharing, experiment operations, or maintaining the CleanRL repository. CleanRL is intentionally script-oriented rather than a modular import library, so route tasks by workflow before recommending commands.

Start Here

1. Read references/install-and-extras.md before installing dependencies, choosing optional extras, or diagnosing import failures.
2. Run python scripts/check_cleanrl_environment.py --check-help in a prepared CleanRL checkout or installed package environment when you need a safe import/help diagnostic.
3. Check references/repo-provenance.md before deciding whether this skill is current for a checkout.
4. Use references/troubleshooting.md for cross-cutting Python, torch, Gym/Gymnasium, W&B, Hugging Face, AWS, and optional-backend failures.

Route by Task

• Training scripts: use sub-skills/training-scripts/SKILL.md to choose a CleanRL algorithm script, inspect tyro/argparse flags, build tiny smoke commands, or adapt single-file training runs.
• Evaluation and sharing: use sub-skills/evaluation-and-sharing/SKILL.md to inspect saved runs/ artifacts, choose evaluation helpers, reason about model-zoo repositories, or guard Hugging Face upload/download actions.
• Experiment operations: use sub-skills/experiment-operations/SKILL.md for benchmark command matrices, W&B tracking, Optuna tuning, resume/reproduce utilities, plotting, Docker, Slurm, Terraform, and AWS Batch planning.
• Repo maintenance: use sub-skills/repo-maintenance/SKILL.md when editing CleanRL code, docs, tests, package metadata, contribution workflows, or RLOps regression plans.

CleanRL Operating Model

• CleanRL's public value is readable, duplicated, single-file algorithm implementations. Do not refactor user tasks into hidden shared abstractions unless the user is deliberately changing repository design.
• Most training entrypoints live under cleanrl/ and are run as scripts, not imported as a stable library API.
• Common script flags include --env-id, --total-timesteps, --seed, --cuda/--no-cuda, --track, --capture-video, --save-model, and --upload-model, but inspect the target script before assuming every flag exists.
• Optional backends are workflow-specific: Atari, EnvPool, Procgen, MuJoCo, DM Control, PettingZoo, JAX, Optuna, cloud/AWS, docs, and memory-gym require separate extras or setup.
• Treat W&B tracking, Hugging Face upload/download, Docker build/push, Terraform, AWS Batch, Slurm submission, and long benchmark/training runs as side-effectful operations that need explicit user approval.

Minimal Checks

python - <<'PY'
import importlib.metadata as metadata
print(metadata.version('cleanrl'))
import cleanrl_utils
print('cleanrl_utils import ok')
PY
python scripts/check_cleanrl_environment.py --check-help

If a check fails, avoid broad dependency installation first. Read references/install-and-extras.md, then install only the extra required by the selected workflow.

Integrated Workflows

• For “train then upload/evaluate,” start in training-scripts to select and run with --save-model; then use evaluation-and-sharing to validate artifacts and handle Hugging Face steps.
• For “benchmark a changed algorithm,” start in repo-maintenance to classify the code change and tests; then use experiment-operations to generate dry-run command matrices or RLOps plans.
• For “cloud training failed,” use experiment-operations for credentials/resources and training-scripts for algorithm flags or backend extras.
• For “new CLI flag,” use repo-maintenance for docs/tests and training-scripts for script flag semantics and tiny smoke commands.

Guardrails

• Do not tell future agents to read or run original CleanRL docs, examples, tests, or benchmark scripts as runtime dependencies. This skill bundles distilled references and safe helper scripts.
• Do not expose tokens, local environment paths, machine-specific install prefixes, or private setup details in user-facing answers.
• Do not run native tests, cloud jobs, W&B jobs, Hugging Face network actions, or long training by default; use the sub-skill safety rules and ask for approval when necessary.