---

name: vllm-omni description: "Use vLLM-Omni for omni-modality model inference and serving: local Omni APIs, OpenAI-compatible --omni servers, stage deploy configs, model recipes, and model integration workflows." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

vLLM-Omni

Use this repo skill when a task involves vLLM-Omni: omni-modality inference or serving for text, image, audio, video, action, diffusion, TTS, OpenAI-compatible APIs, stage-based deployment, supported model recipes, or source-level model integration.

Quick start route

1. For install/import/backend checks, read install-and-backends.md and run the safe checker when needed:

   python scripts/check_environment.py --require-vllm 0.26
   

   Add --require-cuda only when live GPU generation or serving must be verified.

2. Choose the closest sub-skill:

   Task	Read
   Local Python scripts using Omni, AsyncOmni, prompt dictionaries, sampling params, or multimodal outputs	offline-inference
   vllm serve ... --omni, OpenAI-compatible HTTP payloads, curl/OpenAI SDK clients, realtime/streaming, stage head/headless launch	online-serving
   Deploy YAML overlays, stages, connectors, memory placement, --stage-overrides, multi-node/stage planning	stage-configuration
   Choosing supported model families, endpoints, hardware backends, diffusion controls, quantization/offload/cache, benchmark metrics	model-recipes
   Adding custom pipelines, model registrations, TTS adapters, or selecting focused maintainer tests	model-integration

3. If a failure crosses workflows, read troubleshooting.md before running full model examples.

4. To check whether this skill matches the source checkout, read repo-provenance.md.

What vLLM-Omni adds

vLLM-Omni extends vLLM's text-oriented autoregressive runtime with stage-based and diffusion/generation paths for omni-modality workloads:

• multimodal chat and reasoning with text, image, audio, video, and action inputs;
• heterogeneous outputs including text tokens, images, audio, video, latents, trajectories, and custom payloads;
• OpenAI-compatible serving with Omni-specific request fields and endpoints;
• stage-based deploy configs, connectors, head/headless worker launches, and async chunking;
• diffusion image/video/audio pipelines with attention, parallelism, offload, cache, quantization, and LoRA controls;
• TTS and realtime/duplex audio routes;
• model integration surfaces for custom pipelines and supported model families.

Install/backend guardrails

• Use a fresh environment. vLLM, PyTorch, CUDA/ROCm/vendor libraries, media packages, and attention kernels are easy to mix incorrectly.
• Align upstream vllm with the vLLM-Omni release line. For this checkout's docs, vllm==0.26.x is the expected upstream line.
• Set VLLM_OMNI_TARGET_DEVICE during source installs when automation must choose CUDA, ROCm, NPU, XPU, MUSA, or CPU requirements deterministically.
• CPU parser/config checks do not prove live model serving. Full generation usually requires GPU/accelerator hardware, model weights, license/cache access, and runtime budget.
• Do not run broad native examples or benchmarks by default; use bundled no-network helpers first.

Common entry points

• Package import: import vllm_omni for registration side effects and from vllm_omni.entrypoints.omni import Omni for local generation.
• Local API: Omni(model=..., **kwargs).generate(...) and AsyncOmni(model=...).generate(...).
• CLI serving: vllm serve MODEL --omni ... or vllm-omni serve MODEL --omni ....
• Deploy config concepts: DeployConfig, StageDeployConfig, PipelineConfig, stage_id, devices, connectors, gpu_memory_utilization, async_chunk, platform overrides.
• Model/recipe concepts: Qwen3-Omni, Qwen2.5-Omni, Qwen-Image/Z-Image/GLM/Hunyuan/Wan/LTX/Cosmos3/MiniMax-H3, Qwen3-TTS/VoxCPM2/MOSS/Higgs/IndexTTS, DreamZero/GR00T/InternVLA.

Bundled helpers

Helper	Use
scripts/check_environment.py	Verify package metadata, imports, optional CUDA, and Omni CLI help without model downloads.
sub-skills/offline-inference/scripts/build_offline_request.py	Generate safe local Python request snippets without importing vLLM-Omni or loading a model.
sub-skills/online-serving/scripts/build_openai_payload.py	Generate curl/OpenAI SDK payload scaffolds without sending HTTP requests.
sub-skills/stage-configuration/scripts/validate_deploy_yaml.py	Sanity-check deploy YAML shape and connector references without model loading.
sub-skills/stage-configuration/scripts/plan_stage_memory.py	Estimate stage placement and memory utilization from user-provided GPU/stage counts.
sub-skills/model-recipes/scripts/query_model_catalog.py	Query the bundled representative model catalog by task/backend.
sub-skills/model-integration/scripts/check_tts_adapter_contract.py	Statically review TTS adapter source shape without imports or model execution.

When to refresh this skill

Refresh when vLLM-Omni changes public CLI flags, OpenAI protocol fields, deploy schemas, supported model tables, diffusion controls, TTS adapters, model registry behavior, or aligned upstream vLLM version.