---

name: serving-and-cli description: "Launch and operate Xinference services and model lifecycles through public CLI surfaces with safe command templates, placement notes, and troubleshooting handoffs." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Serving and CLI

Use this sub-skill when you need to start a local service, stand up a supervisor/worker cluster, or build safe command lines for model launch and lifecycle management.

Covers

• Local cluster startup with xinference-local.
• Distributed startup with xinference-supervisor and xinference-worker.
• Top-level xinference service commands for:
  • launch
  • list
  • terminate
  • register
  • unregister
  • registrations
  • cached
  • remove-cache
  • engine
  • cal-model-mem
  • vllm-models
  • stop-cluster
  • login
• Placement and launch flags such as:
  • --endpoint / -e
  • --model-name, --model-uid, --model-type
  • --n-worker, --n-gpu, --replica
  • --worker-ip, --gpu-idx
  • --enable-virtual-env, --disable-virtual-env
  • --virtual-env-package, --env
  • --api-key placeholders

Does not cover

• Python client calls or HTTP/OpenAI request bodies -> client-and-api
• Backend or model-family selection, optional engine installs, or model schema design -> models-and-backends
• Auth database, metrics policy, environment policy, or deployment hardening -> operations-and-security

Working pattern

1. Read references/cli-reference.md for exact command shapes.
2. Use scripts/render_xinference_commands.py to generate copy-paste-safe templates.
3. Read references/serving-workflows.md for local, distributed, launch, and shutdown flows.
4. Read references/distributed-deployment.md for host/port and worker placement rules.
5. Read references/troubleshooting.md when a command fails, blocks, or returns a validation error.

Safety rules

• Treat bundled command text as templates, not runnable automation.
• Real launches may download models and per-model dependencies.
• launch requires --model-engine for LLMs.
• Distributed placement must use full registered worker IP:port values.
• --gpu-idx is a comma-separated list of integers.
• Pass engine-specific extras as --key value pairs after the standard launch flags.
• Use --enable-virtual-env or --disable-virtual-env, never both.

Bundled references

• CLI reference
• Serving workflows
• Distributed deployment
• Troubleshooting
• Command renderer