---

name: openchat description: "Use OpenChat/ochat for prompt formatting, OpenAI-compatible serving, and benchmark evaluation of conversational LLMs." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

OpenChat repo skill

Use this skill when the task names OpenChat, ochat, openchat_3.6, openchat_3.5, C-RLFT prompt conditions, OpenChat's OpenAI-compatible server, or OpenChat benchmark/evaluation utilities.

Quick routing

User task	Read next
Choose a model type, serving alias, context length, prompt condition, EOT token, or Hugging Face chat-template behavior	prompting
Build Message/Conversation objects or diagnose ConversationTemplate.tokenize_conversations weights and inference EOT placement	prompting
Start or secure the OpenAI-compatible vLLM/FastAPI server, call /v1/models or /v1/chat/completions, or debug server request errors	serving
Run OpenChat benchmark evaluation, resume partial results, inspect answer matchers, or convert HumanEval outputs to EvalPlus JSONL	evaluation
Fix package import, CUDA, vLLM/Ray, tokenizer-cache, or broad dependency problems before using any workflow	installation and backends and troubleshooting

Training/data generation, DeepSpeed training loops, experimental notebooks, and model-embedding surgery scripts were intentionally excluded from this skill run. Do not route those tasks here unless this skill is refreshed or extended with that scope.

Install and smoke check

OpenChat imports as ochat. A practical setup usually needs Python, PyTorch, Transformers, vLLM/Ray for serving/evaluation, and CUDA-capable hardware for local model serving. Read installation and backends before changing dependencies or deciding whether a CPU-only environment is enough.

Minimal no-download import check:

python scripts/check_openchat_import.py

CUDA/backends check:

python scripts/check_openchat_import.py --check-cuda

These scripts only inspect imports and device availability. They do not download model weights, start a server, call external APIs, or run benchmark suites.

Core OpenChat concepts

• Distribution/import: the package distribution is ochat, and public modules live under ochat.*.
• Model type: canonical keys such as openchat_3.6 and openchat_v3.2_mistral select a conversation template and max context.
• Serving alias: aliases such as openchat_3.5 are accepted by the server after loading the canonical model type; they are not always keys in MODEL_CONFIG_MAP.
• Condition: strings such as GPT4 Correct and Math Correct are prompt-mode labels, not separate models.
• Local runtime: server and local benchmark inference use vLLM/Ray and require suitable model weights plus enough GPU memory.

Evidence and freshness

Read repo provenance before deciding whether this skill matches a current OpenChat checkout. If the source commit, tag, package version, public model registry, server CLI, or evaluation harness changed, run refresh-repo-skill rather than patching stale guidance manually.

Useful bundled helpers

• scripts/check_openchat_import.py verifies core imports, model registry availability, optional serving/evaluation imports, and optional CUDA visibility.
• prompting/scripts/check_prompting_smoke.py runs deterministic prompt/tokenization logic checks without downloading real tokenizers.
• serving/scripts/run_openchat_server.sh safely forwards to the installed server module and refuses non-help launches without --model.
• evaluation/scripts/run_eval.sh forwards to the installed evaluation harness.
• evaluation/scripts/check_answer_matchers.py runs tiny no-model checks for answer extraction behavior.