---

name: litserve description: "Serve custom AI APIs with LitServe, including general LitAPI and LitServer workflows, OpenAI-compatible endpoints, MCP tools, and deployment helpers." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

LitServe

Use this skill when a request mentions LitServe, LitAPI, LitServer, /predict, /v1/chat/completions, /v1/embeddings, /mcp/, client.py, dockerize, health/info/ shutdown routes, batching, streaming, auth, or multi-endpoint serving.

Quick install

• Install the package with pip install litserve.
• Add optional extras only for the workflow you need: fastmcp, Pillow, numpy, openai, httpx, asgi-lifespan, requests, python-multipart, and psutil.
• Verify the installation with scripts/smoke_import.py or a clean python -I -c "import litserve as ls; print(ls.__version__)" check.

Route map

Task family	Read first
General serving, deployment, payloads, auth, middleware, clients, Docker	sub-skills/server-basics/SKILL.md
OpenAI-compatible chat completions and embeddings	sub-skills/openai-specs/SKILL.md
MCP tool exposure and streamable HTTP mounting	sub-skills/mcp/SKILL.md

Reference map

• references/installation.md: read for install options, optional extras, and smoke checks.
• references/troubleshooting.md: read for package-wide import, CLI, payload, auth, and deployment failures.
• references/repo-provenance.md: read when you need the source snapshot and staleness baseline.
• references/repo-routing-metadata.json: structured metadata for repo-skills-router import.
• scripts/smoke_import.py: run to print the import smoke report and core signatures.
• scripts/smoke_server.py: run to launch the bundled minimal server and verify one request end to end.

Shared notes

• The root skill is a router. Detailed workflows, validation tables, and troubleshooting belong in the subskills and bundled references.
• Do not point runtime instructions back to source-checkout paths.
• Prefer the root smoke script and the bundled subskill scripts before reopening the original repository.

Good starting questions

• How do I serve a custom model or pipeline?
• How do I add batching, streaming, auth, or middleware?
• How do I expose an OpenAI-compatible chat or embedding endpoint?
• How do I expose an MCP tool from a LitAPI?
• How do I package a LitServe app for Docker or local deployment?