---

name: serving-deployment description: "Route Qwen CLI, web UI, OpenAI-compatible API, vLLM, FastChat, Docker, TensorRT, Ascend, and DCU serving or deployment workflows." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Qwen Serving and Deployment

Use this sub-skill when the user wants to run Qwen as an interactive CLI, Gradio web demo, OpenAI-compatible FastAPI server, vLLM/FastChat service, Docker container, TensorRT deployment, Ascend NPU workflow, or Hygon DCU/fastllm deployment.

Safe start

• Use scripts/qwen_launch_advisor.py to build local CLI/Web/API commands without loading a checkpoint or opening a port.
• Use scripts/qwen_docker_command_builder.py to build Docker commands without pulling images or running containers.
• Validate local checkpoint directories before recommending a service launch; Docker scripts and local loaders expect config.json plus tokenizer/model assets.
• Bind to 127.0.0.1 by default. Use 0.0.0.0, --share, or Docker port publishing only when the user accepts network exposure.

Routes

User request	Read
cli_demo.py, web_demo.py, openai_api.py, flags, endpoint semantics, streaming, BasicAuth, request schemas	references/local-demos-and-api.md
vLLM worker, FastChat controller/API server, standalone vLLM OpenAI API, tensor parallel, dtype, ChatML template	references/vllm-fastchat.md
qwenllm/qwen Docker images, checkpoint mounts, web/API/CLI container commands, logs, cleanup	references/docker-deployment.md
TensorRT-LLM, Ascend NPU, Hygon DCU, fastllm conversion, vendor device prerequisites	references/accelerator-support.md
Port, auth, checkpoint, Docker, GPU, function-calling, public-share, or vendor-runtime failures	references/troubleshooting.md

Boundaries

• For raw model loading or batch inference code, use ../inference-model-loading/SKILL.md.
• For ReAct/function-calling message conversion and tokenizer/ChatML details, use ../prompting-tool-use-tokenization/SKILL.md.
• For fine-tuning or GPTQ creation, use ../finetuning-quantization/SKILL.md.
• For benchmark scripts or repo tests, use ../evaluation-reproduction/SKILL.md.

Side-effect rules

Never start a long-running server, pull a Docker image, publish a Gradio share link, run a container, launch vLLM, or mount a vendor device as a smoke test. First provide a dry-run command, list prerequisites, and ask/confirm when the user wants the side effect.