---

name: server-basics description: "Serve models and custom pipelines with LitServe LitAPI and LitServer, including batching, streaming, auth, callbacks, loggers, middleware, payloads, multi-endpoints, clients, Docker, and deployment." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

server-basics

Use this sub-skill when a task asks how to build, run, deploy, or debug a general LitServe server. It covers the standard LitAPI + LitServer workflow and the HTTP-serving surfaces around it.

Use this for

• Serving a custom model or pipeline with litserve.LitAPI and litserve.LitServer.
• Choosing accelerator, devices, workers_per_device, timeout, num_api_servers, host, port, logging, and client-generation settings.
• Adding batching, streaming, async handlers, auth, callbacks, loggers, middleware, request tracking, payload limits, health/info/shutdown endpoints, and multiple API paths.
• Handling JSON, form data, multipart file uploads, and base64 image input/output.
• Generating a local Dockerfile with litserve dockerize or deploying through the Lightning CLI passthrough.

Route elsewhere

• OpenAI-compatible chat completions or embeddings belong in ../openai-specs/SKILL.md.
• MCP tool exposure and MCP server mounting belong in ../mcp/SKILL.md.
• Torch/CUDA benchmark harnesses, FastAPI parity tests, throughput benchmark scripts, and transformer/vision benchmark reproduction are intentionally out of scope.
• Start from the root LitServe router when the user request spans multiple sub-skills: ../../SKILL.md.

Reference map

Read these bundled references instead of opening the source repository:

• references/api-reference.md: constructor defaults, hooks, endpoint registration, auth, callbacks, loggers, middleware, client generation, and Docker/deployment knobs.
• references/workflows.md: task recipes for serving, batching, streaming, async, files, images, auth, multi-endpoints, client generation, Docker, and deployment.
• references/data-formats.md: JSON, form, multipart, Pydantic, image, and streaming payload contracts.
• references/troubleshooting.md: common validation errors, readiness issues, auth mismatches, client/Docker problems, and excluded paths.

Bundled examples

• scripts/minimal_server.py: safe CPU square server with port, host, accelerator, batching, workers, and client-generation options.
• scripts/file_upload_server.py: multipart file and form-data server with an optional payload-size limit.
• scripts/image_roundtrip_server.py: base64 image request/response server using ImageInput and ImageOutput.

Operating notes

• Prefer putting max_batch_size, batch_timeout, api_path, stream, loop, spec, mcp, and enable_async on the LitAPI constructor. The same settings on LitServer are deprecated compatibility paths.
• Load heavy model state in setup(device), not in __init__.
• For local development use host="127.0.0.1"; for containers and remote deployment use host="0.0.0.0".
• Do not use this sub-skill to implement OpenAI or MCP specs directly; route those to the sibling sub-skills so their endpoint contracts remain precise.