---

name: rwkv-lm description: "Routes RWKV-LM repository tasks for RWKV training data, RWKV-7 inference/evaluation, architecture comparison, and RWKV-8 ROSA experiments." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

RWKV-LM

Use this skill for tasks involving the RWKV-LM repository, RWKV-5/6/7/8 model scripts, RWKV recurrent language-model training, RWKV token/binidx data, RWKV-7 Goose inference, Qwen/RWKV tensor comparison, or RWKV-8 ROSA research prototypes.

RWKV-LM is a script-oriented research repository, not a single installable Python package. Future agents should use this skill to choose the right repository workflow and avoid mixing version directories, tokenizer families, training flags, and backend assumptions.

Quick routing

User task	Read
Convert JSONL to RWKV .bin/.idx, compute magic_prime, or construct MiniPile/Pile training commands	training-data
Decide between RWKV-7 GPT-mode, RNN-mode, fast inference, or MMLU-style evaluation	inference-evaluation
Explain RWKV-7 tensor names, state shapes, Qwen3.5 export, or context-parallel state composition	architecture-reference
Understand RWKV-8/Heron/ROSA toy scripts, reverse-digit demos, or suffix-automaton behavior	rosa-experiments
Debug install, backend, tokenizer, data, or checkpoint confusion across workflows	troubleshooting.md
Check whether this skill is current for a checkout	repo-provenance.md
Understand version-directory history	compatibility-and-history.md

Minimal environment guidance

For most inspection, prompt rendering, data conversion, and architecture tasks:

python -m pip install torch numpy rwkv transformers datasets safetensors

For RWKV-7 training commands, the repository recommends Python 3.10+, PyTorch 2.5+ or newer, CUDA 12.5+ or a compatible wheel, current DeepSpeed, ninja, wandb if logging, and pytorch-lightning==1.9.5. Do not upgrade Lightning blindly; the training scripts branch on the Lightning version.

Run the bundled runtime check from this skill tree:

python scripts/check_runtime.py --check-cuda

A passing CUDA allocation proves PyTorch can see the GPU. It does not prove that RWKV-LM custom CUDA kernels can compile; check CUDA_HOME, nvcc, and the specific directory's cuda/ sources for training or fast inference claims.

Important repository conventions

• RWKV-7 RWKV-v7/train_temp is the current reference training implementation.
• RWKV-6 uses the RWKV-v5 tree with --my_testing x060.
• RWKV-7 standard examples use the rwkv_vocab_v20230424 tokenizer and often vocab_size 65536.
• Training data is a prefix pair (<prefix>.bin, <prefix>.idx) passed to the trainer without a suffix.
• magic_prime depends on token count and context length. Recompute it for every corpus or ctx_len change.
• Stage-2/3 training resumes by scanning proj_dir for rwkv-*.pth; inspect checkpoint names before resuming.
• Demo scripts in the repository contain maintainer-local absolute checkpoint paths. Treat those as placeholders and require user-provided paths.

Safety boundaries

Do not start network downloads, model checkpoint downloads, full training, MMLU evaluation, or CUDA source-extension builds unless the user explicitly requests that operation and the environment is ready. Many native repo examples are long-running, data-dependent, GPU-dependent, or checkpoint-dependent. When the user only needs planning, conversion, prompt rendering, or troubleshooting, use the bundled references/scripts here instead of running original examples.

Refresh and provenance

Before using this skill for a different checkout, compare the current commit, major version directories, and public training/inference scripts against repo-provenance.md. Run refresh-repo-skill if the checkout changed in a way that may affect flags, tokenizers, model shapes, or backend requirements.