---

name: chatllama-rlhf description: "Guides ChatLLaMA dataset preparation, config validation, actor/reward/RLHF training, and troubleshooting for source-era RLHF workflows." disable-model-invocation: true metadata: disco-role: operating license: NO_LICENSE

ChatLLaMA RLHF

Use this sub-skill when the user wants to prepare ChatLLaMA datasets, validate the config YAML, train the actor/reward/RLHF stages, or debug the source-era dependency stack.

Triggers

• Prepare actor_training_data.json, reward_training_data.json, or rlhf_training_data.json.
• Validate config.yaml, DeepSpeed config, or PEFT config files.
• Run generate_rewards.py or main.py --type ACTOR|REWARD|RL|ALL.
• Debug model-family support, checkpointing, or pin drift in the RLHF stack.

Read next

• references/api-reference.md for the public config, dataset, loader, and training entry points.
• references/data-and-config.md for the YAML sections, JSON dataset schemas, and checkpoint layout.
• references/workflows.md for the dataset-prep and training sequence.
• references/troubleshooting.md for import, pin, checkpoint, and model-family failures.
• scripts/chatllama_rlhf_probe.py for a safe config and dataset schema check.

What to include

• Config, ConfigActor, ConfigReward, ConfigTrainer, and the ConfigCritic alias.
• Dataset preparation and reward-generation helpers.
• Actor, reward, critic, and RL training orchestration.
• DeepSpeed / Accelerate / PEFT compatibility notes.
• The source-era Python and dependency constraints.

What to exclude

• External weight downloads and live API calls by default.
• Full training runs as the first action.
• Generic Transformers fine-tuning outside the ChatLLaMA pipeline.

Quick decision rule

If the user mentions actor, reward, RLHF, or config.yaml, start here. If the question is only about generic LLM serving or inference, route to the root skill only if the task still depends on ChatLLaMA artifacts.