---

name: inference description: "Operate and troubleshoot OpenAssistant inference server, workers, text client, safety server, model configs, websocket work protocol, and SSE chat flows." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

OpenAssistant inference sub-skill

Use this sub-skill when a task is about the OpenAssistant local inference stack: the FastAPI inference server, websocket workers, text client, safety server, model configuration selection, chat streaming, plugin calls, GPU sizing, or lightweight _lorem/distilgpt2 smoke checks.

Route the request

• Server/worker/chat protocol, debug auth, route map, websocket request/response types, SSE events, plugins, or safety parameters: read references/api-reference.md.
• Model config names, _lorem, distilgpt2, OpenAssistant SFT/RLHF variants, quantization, or GPU memory sizing: read references/model-configs.md and run scripts/check_inference_config.py when a checkout is available.
• Local Docker compose inference stack, text client, worker startup, full dev tmux flow, safety server, or load testing: read references/workflows.md.
• Failures involving API keys, websocket status, protocol upgrade, Redis/Postgres, TGI connection, model downloads, GPU OOM, SSE parsing, plugin OpenAPI, or safety server availability: read references/troubleshooting.md.

Safe checks

These checks do not download models or start services:

python scripts/check_inference_config.py --repo-root <repo-root> --model-config _lorem
python scripts/check_inference_config.py --repo-root <repo-root> --model-config distilgpt2 --json
python scripts/check_inference_config.py --repo-root <repo-root> --list

Use _lorem when the user needs a CPU-only logical smoke path. Use distilgpt2 only when a tiny Hugging Face model download/cache is acceptable or already available. Real OpenAssistant model configs may require large downloads and GPU memory.

Boundaries

• Route website chat components, browser-side API calls, and frontend SSE rendering to the website sub-skill.
• Route the data-collection backend, task lifecycle, shared REST API client, and OA JSONL exports to the backend sub-skill.
• Model training/evaluation/pretokenization, production deployment, Ansible, and infrastructure operations are excluded from this generated skill run.
• Do not run a real model worker, download model weights, use credentials, or start Docker services unless the user explicitly requests it and accepts resource/network requirements.