---

name: serving description: "Routes MOSS serving and UI deployment tasks for FastAPI payloads, Gradio or Streamlit demos, session history, and service troubleshooting." disable-model-invocation: true metadata: disco-role: operating license: NOASSERTION

MOSS serving and UI

Use this sub-skill when a task asks how to expose MOSS through an API or UI, prepare a FastAPI request payload, understand Gradio/Streamlit controls, or troubleshoot service startup and request handling.

Read this when

• The user mentions moss_api_demo.py, FastAPI, Uvicorn, POST /, uid, history, max_length, top_p, or temperature.
• The task asks for a Gradio or Streamlit MOSS chat UI.
• You need a safe curl payload or request template without contacting a server.
• You are debugging port conflicts, JSON request shape, session-history reuse, UI dependency imports, CUDA OOM, or slow model loading.

Route elsewhere

• For prompt syntax and generation parameters, read ../inference/SKILL.md.
• For model class imports, checkpoint families, quantization, and CUDA smoke, read ../model-runtime/SKILL.md.
• For fine-tuning or data validation, read ../fine-tuning-data/SKILL.md.
• For shared installation requirements, read ../../references/install-and-dependencies.md.

Operating workflow

1. Choose service type. Use FastAPI when the user needs programmatic POST calls; use Gradio for a browser chat demo with sliders; use Streamlit for a sidebar-controlled local app. All full services load a MOSS checkpoint and require the same model/backend planning as inference.
2. Prepare request shape before launch. The FastAPI demo expects JSON fields prompt, optional uid, optional max_length, optional top_p, and optional temperature. It returns response, history, status, time, and uid.
3. Keep state semantics clear. The API uses uid to preserve history; the Gradio and Streamlit demos maintain history in UI state. A new or missing uid starts a fresh API conversation.
4. Generate safe payloads with the bundled helper. Run scripts/moss_request_template.py to validate values and print JSON/curl snippets without contacting a server.
5. Launch only after model readiness is explicit. A real service may download large checkpoints and allocate GPU memory before accepting requests. Check dependencies, model path/cache, GPU selection, and port availability.

Safe helper example

python path/to/moss/sub-skills/serving/scripts/moss_request_template.py \
  --prompt "Hello MOSS" --max-length 512 --top-p 0.8 --temperature 0.7 --curl

The helper prints a payload and curl snippet. It does not send traffic and does not require a running MOSS service.

References

• references/workflows.md — FastAPI request/response schema, UI behavior, startup planning, and validation checklist.
• references/troubleshooting.md — API, port, dependency, CUDA, model download, UI rendering, and state-history failures.
• scripts/moss_request_template.py — safe JSON payload/curl generator.

Answering checklist

• State which demo surface is being used: FastAPI, Gradio, or Streamlit.
• Include the request fields and response fields for API tasks.
• Warn when a command will start a service, download a checkpoint, or allocate GPU memory.
• Preserve uid when continuing an API conversation; omit or generate it for a fresh session.
• Do not claim the server is live unless the task explicitly launched and checked it.