---

name: llm-fine-tuning description: "Use AgileRL LLM fine-tuning and post-training workflows for GRPO, CISPO, GSPO, DPO, SFT, LLM PPO/REINFORCE, vLLM, DeepSpeed, and optional LLM dependencies." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

AgileRL LLM Fine-Tuning

Use this sub-skill when the task involves AgileRL LLM post-training or fine-tuning: GRPO, CISPO, GSPO, LLM PPO, LLM REINFORCE, SFT, DPO, reasoning/preference/multiturn/SFT trainers, AgileRL LLM envs, vLLM rollout backends, DeepSpeed, PEFT, quantization, checkpointing, or agilerl[llm].

Read First

• references/workflows.md for reasoning, preference, multiturn, and SFT/DPO routes.
• references/api-reference.md for algorithms, trainers, envs, and utilities.
• references/configuration.md for optional dependencies, Accelerate, vLLM, quantization, and checkpoint settings.
• references/troubleshooting.md for backend, model, tokenizer, reward, and dependency failures.
• scripts/inspect_llm_dependencies.py --help for safe optional dependency checks.

Boundaries

• Use ../hpo-and-mutation/SKILL.md for generic mutation/tournament concepts; this sub-skill explains LLM-specific limitations.
• Use ../training-workflows/SKILL.md for classical Gymnasium RL.
• Use ../offline-bandits-data/SKILL.md for tabular/offline RL datasets, not LLM preference/reasoning data.

Workflow Routes

Task	AgileRL route
Reasoning RL with verifiable rewards	GRPO/CISPO/GSPO and finetune_llm_reasoning(...)
Preference optimization	DPO or preference-oriented GRPO and finetune_llm_preference(...)
Multi-turn LLM agents	LLM PPO, LLM REINFORCE, GRPO, and finetune_llm_multiturn(...)
Supervised fine-tuning	SFT and finetune_llm_sft(...)
Checkpoint/quantization/vLLM rollout planning	Configuration and troubleshooting references here

Safe Validation

python scripts/inspect_llm_dependencies.py --json

This reports optional packages and CUDA availability. It does not download models, load vLLM, start DeepSpeed, or train.