---

name: openllm description: "Guides OpenLLM installation, model catalog management, local OpenAI-compatible serving, BentoCloud deployment, and operational troubleshooting for self-hosted LLM workflows." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

OpenLLM Repo Skill

Use this skill when a task asks about OpenLLM or about running open-source LLMs through an OpenAI-compatible local server, terminal chat, custom model repositories, or BentoCloud deployment with the openllm CLI.

OpenLLM is a Typer-based Python CLI. Its public entry point is openllm, and the package distribution is openllm.

Fast install and import check

For ordinary use, install the package from PyPI:

python -m pip install openllm
openllm --help
openllm --version

For a source checkout, use an editable install in an isolated environment:

python -m pip install -e .
python -c "import openllm; print(openllm.__name__)"
openllm --help

Read references/repo-provenance.md before deciding whether this generated skill matches a different checkout or needs refresh.

Route by task

Task or signal	Read next
Start an LLM server, use /chat, call the OpenAI-compatible /v1 API, use openllm serve, openllm run, or openllm hello	sub-skills/local-serving/SKILL.md
List models, inspect a model, update/add/remove model repositories, parse custom repo URLs, or debug missing model catalog entries	sub-skills/model-repositories/SKILL.md
Deploy a model to BentoCloud with openllm deploy, choose an instance type, pass --env, or debug BentoCloud login/context/config	sub-skills/cloud-deployment/SKILL.md
Diagnose install/import, GPU/resource detection, per-model dependency venvs, cache locations, cleanup commands, or analytics opt-out	sub-skills/environment-maintenance/SKILL.md
Need a one-page root command map	references/cli-overview.md
Cross-cutting install, network, credentials, hardware, or cache failures	references/troubleshooting.md

Core CLI families

openllm hello                 # interactive starter that updates model repos and suggests run/serve/deploy
openllm serve MODEL[:VERSION] # local OpenAI-compatible server and browser chat UI
openllm run MODEL[:VERSION]   # local server plus terminal chat loop
openllm deploy MODEL[:VERSION]# deploy to BentoCloud
openllm model list|get        # inspect available Bentos/models
openllm repo list|add|remove|update|default
openllm clean ...             # remove model, repo, venv, or config caches; some commands are destructive

OpenLLM model serving is model-specific: full serving may require a model repository cache, model downloads, per-Bento dependencies, Hugging Face credentials for gated models, and enough GPU/CPU resources. Use this skill's dry-run helpers and troubleshooting references before starting long-running downloads or deployments.

Safe bundled helpers

• Run scripts/check_openllm_install.py to check Python importability, distribution version, CLI availability, and optional NVIDIA GPU visibility without starting a model.
• Use sub-skill scripts for workflow-specific dry runs, command planning, local catalog inspection, and resource estimation.

Boundaries

This is an operating skill for using OpenLLM as a package and CLI. It is not a guide to editing OpenLLM's release automation, pushing tags, or mutating the original repository. Do not run cleanup, deployment, repository cloning, or model download commands unless the user explicitly asks for that side effect and understands the credentials/network/storage implications.