---

name: model-runtime description: "Routes MOSS model-runtime tasks for config, tokenizer, causal language model classes, checkpoint families, quantization, and backend readiness." disable-model-invocation: true metadata: disco-role: operating license: NOASSERTION

MOSS model runtime

Use this sub-skill when the task is about understanding or validating the MOSS runtime components rather than chatting, serving, or fine-tuning. It covers the Hugging Face-style PyTorch implementation in models/, the optional quantized runtime, and the model/checkpoint constraints that determine whether a workflow can run locally.

Read this when

• A task names MossConfig, MossTokenizer, MossModel, or MossForCausalLM.
• You need to choose between moss-moon-003-sft, moss-moon-003-sft-int8, and moss-moon-003-sft-int4 for memory, backend, or quantization reasons.
• You need to verify imports or CUDA readiness without downloading a 16B checkpoint.
• You are debugging trust_remote_code, Triton/GPTQ, model-parallel loading, missing tokenizer files, or checkpoint memory failures.
• You need source-backed architecture facts before writing inference, serving, or training code.

Route elsewhere

• For prompt markers, generation defaults, CLI command construction, or Jittor chat use, read ../inference/SKILL.md.
• For FastAPI, Gradio, and Streamlit services, read ../serving/SKILL.md.
• For SFT data format, no-loss spans, and DeepSpeed fine-tuning, read ../fine-tuning-data/SKILL.md.
• For cross-cutting install guidance and model catalog context, read ../../references/install-and-dependencies.md and ../../references/model-overview.md.

Operating workflow

1. Identify the runtime surface. MOSS exposes a source-checkout runtime, not a conventional installable Python distribution. For local checkout work, ensure the checkout root is importable so models.configuration_moss, models.modeling_moss, and models.tokenization_moss resolve.
2. Check the configuration before allocating weights. Read references/model-architecture.md for the verified defaults: 107008 vocabulary entries, 2048 context length, 4096 hidden size, 28 layers, 16 heads, rotary dimension 64, <eom> EOS id 106068, and quantization fields wbits/groupsize.
3. Select the checkpoint family. Use FP16 moss-moon-003-sft for model parallelism; use *-int4 or *-int8 only on one GPU. Quantized models use GPTQ/Triton support and are documented as not supporting model parallelism.
4. Verify imports safely. Run the bundled scripts/check_model_runtime.py with a local checkout only when you need runtime evidence. It imports classes, instantiates a tiny model from a small config, and optionally checks CUDA without calling from_pretrained.
5. Escalate to real checkpoint loading only when required. Loading actual MOSS checkpoints can download large model files and require substantial GPU memory. Treat that as a task-specific execution decision, not a default diagnostic.

Safe validation commands

From any working directory with the required dependencies installed:

python path/to/moss/sub-skills/model-runtime/scripts/check_model_runtime.py --help
python path/to/moss/sub-skills/model-runtime/scripts/check_model_runtime.py --repo-root /path/to/MOSS --json
python path/to/moss/sub-skills/model-runtime/scripts/check_model_runtime.py --repo-root /path/to/MOSS --cuda --json

The helper accepts a user-supplied checkout root. It does not rely on the checkout used to generate this skill and does not download checkpoints.

References

• references/model-architecture.md — class roles, verified configuration defaults, tokenizer behavior, checkpoint families, and quantization notes.
• references/troubleshooting.md — import, dependency, CUDA, quantization, checkpoint, and memory failure recovery.
• scripts/check_model_runtime.py — safe import/signature/CUDA diagnostic helper.

Quality checks before answering a task

• Do not claim a full MOSS checkpoint works unless it was actually loaded in the target environment.
• Do not use a CPU import check as proof of CUDA generation, quantized Triton kernels, model parallelism, or fine-tuning.
• Do not recommend multi-GPU model parallelism for *-int4 or *-int8 checkpoints.
• Do not tell users to run source demo scripts as the only guidance; use this skill's bundled references and helpers to construct safe commands first.