---

name: workload-management description: "Operate Lepton endpoint, pod, job, Ray cluster, fine-tune, template, node, log, event, and workload status workflows from safe CLI and SDK plans." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Workload Management

Use this sub-skill when a task involves Lepton compute workloads: endpoint/deployment serving, dev pods, batch jobs, Ray clusters, fine-tuning jobs, templates, resource shapes, node capacity, workload logs, events, status checks, restarts, stops, starts, and removal plans.

Do not run live cloud mutations unless the user has explicitly asked for them and the root or cli-operations safety route has handled confirmation. This sub-skill supplies workload-specific commands and validation checks; generic workspace authentication belongs to workspace-and-auth, and generic command confirmation/output handling belongs to cli-operations.

Choose the route

• Read references/workload-taxonomy.md when the user is choosing between endpoint, dev pod, batch job, Ray cluster, and fine-tuning job.
• Read references/endpoint-and-pod-workflows.md for lep endpoint/legacy lep deployment and lep pod create/list/get/status/update/stop/restart/remove/log/SSH flows.
• Read references/job-ray-finetune-workflows.md for batch job, Ray cluster, Ray job, fine-tune job, and template workflows.
• Read references/logs-events-nodes.md for historical logs, streaming logs, events, replicas, nodes, node groups, resource shapes, and node-group storage visibility.
• Read references/troubleshooting.md when create/update/list/log/SSH/status commands fail or produce surprising no-op behavior.
• Use scripts/build_workload_command.py to produce dry-run command examples for endpoint, job, and pod basics without executing lep.

Workload task mapping

User intent	Best workload route	First read-only command
Serve an HTTP model, embedding service, or custom container continuously	Endpoint	lep endpoint list then lep endpoint status -n NAME
Open a personal interactive machine, notebook, SSH shell, or IDE target	Dev pod	lep pod list --detail then lep pod get -n NAME
Run training, evaluation, processing, or sweeps to completion	Batch job	lep job list then lep job get --id JOB_ID
Run distributed Python/Ray pipelines with a head and worker groups	Ray cluster	lep raycluster list then lep raycluster get -n NAME
Launch guided model fine-tuning from a trainer/template schema	Fine-tune job	lep finetune list then lep finetune get -i JOB_ID
Find valid shapes/capacity before creating workloads	Nodes/shapes	`lep node resource-shape --purpose deployment
Diagnose workload output or failures	Logs/events	lep log get ..., lep endpoint events -n NAME, or lep job events -i JOB_ID

Safe workload workflow

1. Identify the workload type. If the user says "service", "API", "serve", "replicas", "QPM", or "scale to zero", prefer endpoint. If they say "SSH", "Jupyter", "debug machine", or "interactive", prefer pod. If they say "run once", "training job", "sweep", or "completion", prefer job. If they say "Ray actors", "Ray job", "head/worker", or "autoscaling Ray", prefer Ray cluster. If they say "fine-tune", "trainer", "base model", or "dataset recipe", prefer fine-tune.
2. Gather read-only state first. Use list/get/status commands and shape/node discovery before proposing create/update/stop/delete commands.
3. Plan mutations as commands, not execution. Show the exact lep command, expected side effects, required workspace context, and whether it may create cost, stop compute, delete resources, or replace fields.
4. Validate workload-specific arguments. Check create/update flags for container image/name, resource shape, port syntax, autoscale exclusivity, header-routing booleans, job filters, Ray worker group structure, and log time/replica scope.
5. Use scoped diagnostics. Prefer status/get, replicas/nodes, events, and time-scoped logs over broad log downloads or unscoped destructive operations.

Core command families

• Endpoint/deployment: lep endpoint create|list|get|status|update|stop|restart|remove|log|events; the hidden legacy alias lep deployment ... maps to the same command family for compatibility, but prefer lep endpoint in new plans.
• Dev pod: lep pod create|list|get|ssh|stop|remove; use lep pod list --detail for SSH/TCP/JupyterLab columns and lep pod get -n NAME for JSON details.
• Batch job: lep job create|list|get|stop|start|remove|remove-all|stop-all|clone|log|replicas|nodes|events.
• Ray cluster: lep raycluster create|list|get|update|start|stop|remove|submit-job|list-jobs|stop-job.
• Fine-tune: lep finetune list|get|create|delete|list-trainers; create dynamically injects trainer flags from a trainer template unless --file mode is used.
• Templates and capacity: lep template list, hidden lep template get, lep node list, lep node list-nodes, lep node list-reservations, lep node resource-shape, lep node storage.
• Historical logs: lep log get --endpoint NAME ..., lep log get --job JOB_ID ..., or lep log get --job-name NAME ...; use direct lep endpoint log / lep job log for replica live-stream style commands.

Python SDK surfaces

For scripts that need API objects instead of shell commands, use APIClient() after workspace context is configured by the auth route. The workload attributes are:

• client.deployment: endpoint/deployment API, dispatching to the new endpoint implementation when enabled.
• client.pod: dev pod API, dispatching to the new devpod implementation when enabled.
• client.job, client.raycluster, client.finetune, client.template, client.shapes, client.nodegroup, and client.log for the matching resource families.

Keep SDK examples read-only unless the user has explicitly requested a mutation. Use .safe_json(...) or model serialization for reviewable payloads before create/update/delete.

Boundaries and cross-routes

• Storage/file transfer, secrets, ingress/canary routing, IP allowlist design, and storage mount validation are owned by storage-secrets-ingress; this sub-skill only notes workload flags that reference mounts, secrets, tokens, or IP allowlists.
• Authentication, workspace selection, environment variables, and token redaction are owned by workspace-and-auth.
• Generic confirmation rules, command tree discovery, output capture, and destructive-command confirmation are owned by root/cli-operations; apply them before running any create/update/stop/start/remove/delete command.