---

name: evaluation description: "Routes Chinese-LLaMA-Alpaca-2 C-Eval, CMMLU, and LongBench evaluation workflows." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

evaluation

Use this sub-skill when the task is about benchmark evaluation, result files, or benchmark-specific prompt/scoring logic.

Use it when

• the user mentions C-Eval, CMMLU, LongBench, LongBench-E, or benchmark result JSON/CSV files
• the task is about few_shot, cot, with_prompt, constrained_decoding, or per-subject accuracy
• the task needs LongBench prediction and scoring behavior
• the user asks how to interpret summary files produced by the repo scripts

Workflow

1. Read references/workflows.md to select C-Eval, CMMLU, LongBench prediction, or LongBench scoring.
2. Confirm the expected benchmark data layout and model path before running a heavy evaluation.
3. Use a fresh output directory per evaluation run.
4. Read references/troubleshooting.md if data directories, subject metadata, prompt config, or output files are missing.

Bundled runtime files

• scripts/ceval/eval.py
• scripts/ceval/evaluator.py
• scripts/ceval/llama_evaluator.py
• scripts/ceval/subject_mapping.json
• scripts/cmmlu/eval.py
• scripts/cmmlu/evaluator.py
• scripts/cmmlu/llama2_evaluator.py
• scripts/cmmlu/categories.py
• scripts/longbench/pred_llama2.py
• scripts/longbench/eval.py
• scripts/longbench/metrics.py
• scripts/longbench/config/
• scripts/attn_and_long_ctx_patches.py

What to read first

• references/workflows.md for benchmark-specific command shapes and output files
• references/troubleshooting.md for data layout and optional dependency issues

Routing notes

• Use this sub-skill for benchmark execution and scoring only.
• Use hf-inference for ad hoc generation outside the benchmark scripts.
• Use train-and-merge if the user first needs to produce or merge the model being evaluated.