---

name: inference-pipelines description: "Use Transformers pipelines and AutoClass inference workflows across text, vision, audio, video, and multimodal tasks, with safe device, dtype, batching, offline, and validation choices." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Inference Pipelines

Use this sub-skill when an agent needs high-level inference with transformers.pipeline() or needs to decide whether a Pipeline, AutoClass call, or lower-level generation workflow is the right fit.

Start Here

• Choose Pipeline for standard inference tasks where Transformers owns preprocessing, model forwarding, and postprocessing.
• Choose AutoClass inference when the task needs custom tensor logic, custom preprocessing, or nonstandard outputs not exposed by a task pipeline.
• Choose the sibling generation sub-skill when the main challenge is chat templating, generate(), streamers, decoding, generation config, or token-level control.
• Choose the sibling tokenizers-processors sub-skill when the main challenge is tokenizer/processor loading, padding/truncation, chat templates, image/audio processors, or multimodal preprocessing.
• Choose the sibling serving-cli sub-skill for CLI, server, deployment, or command-entrypoint workflows.
• Choose the sibling quantization-integrations sub-skill when memory reduction, bitsandbytes, Accelerate placement, or third-party quantization drives the design.
• Choose the sibling model-extension sub-skill for custom models, new architectures, pipeline registration, or trust_remote_code review.

Core API

The inspected package exposes transformers.pipeline() with these important constructor choices:

from transformers import pipeline

pipe = pipeline(
    task="text-classification",
    model="distilbert/distilbert-base-uncased-finetuned-sst-2-english",
    device=-1,
    dtype="auto",
)
print(pipe("Transformers pipelines are convenient."))

Key parameters include task, model, config, tokenizer, feature_extractor, image_processor, video_processor, processor, revision, use_fast, token, device, device_map, dtype="auto", trust_remote_code, model_kwargs, pipeline_class, and task-specific **kwargs.

Use references/api-reference.md for task families, constructor parameters, expected input/output shapes, AutoClass alternatives, and validation checks.

Fast Workflow

1. Identify the modality and task: text, image, audio, video, document, or multimodal.
2. Pick a concrete task id and model id/local path; avoid relying on default models for reproducible code.
3. Decide model source:
   • Hub id for online examples and normal usage.
   • Local directory for offline, pinned, or private model files.
   • revision for reproducible Hub loading.
   • local_files_only=True for no-network validation.
4. Decide hardware:
   • CPU: device=-1 or omit device; avoid batching unless measured.
   • One GPU: device=0; consider dtype="float16" only when supported.
   • Apple Silicon: device="mps"; validate dtype support.
   • Automatic multi-device placement: use device_map="auto" and do not also set device.
5. Build the pipeline and run one tiny representative input.
6. Assert output shape and keys before integrating into application code.
7. Add batching only after correctness is proven and memory behavior is measured.

Use references/workflows.md for complete patterns and copyable snippets.

Safe Offline Smoke

This sub-skill bundles a dry-run-first helper:

python sub-skills/inference-pipelines/scripts/pipeline_smoke.py \
  --task text-classification \
  --model ./local-model \
  --local-files-only

By default the script imports Transformers, checks task/model arguments, tries AutoConfig.from_pretrained(..., local_files_only=True) when a model is provided, and prints construction advice without downloading weights. Add --run only when the requested model and optional dependencies are available locally or network access is intended.

Common Decisions

• Prefer dtype="auto" for generic code; use explicit float16/bfloat16 only after confirming the backend and hardware support it.
• Do not pass both device and device_map; pipeline warns that device overrides device_map and can produce unexpected behavior.
• Use model_kwargs={...} for model-loading options, but do not duplicate dtype or device_map both top-level and inside model_kwargs.
• Use token=True or a token string only for private/gated Hub models; never hard-code secrets in skill examples or application code.
• Keep trust_remote_code=False unless the model repository code has been reviewed and the workflow explicitly requires it.
• For offline code, set local_files_only=True and point model, tokenizer, processor, and config arguments to local directories.

Output Validation

Validate outputs by task rather than assuming one shape:

• Text classification: list of dictionaries with label and score.
• Text generation: list of dictionaries with generated_text; chat inputs may return generated message structures depending on task/model behavior.
• Token classification: token/span dictionaries; aggregation settings affect shape.
• Question answering: dictionary with answer, score, start, and end when supported.
• Image/audio/video classification: list of label/score dictionaries.
• Object detection and segmentation: dictionaries can include boxes, masks, labels, and scores.
• ASR: dictionary with text, optionally chunks when timestamps are requested.
• Feature extraction: nested numeric arrays or tensors depending on options.

Troubleshooting Routes

Use references/troubleshooting.md when you see:

• Optional dependency errors for PyTorch, torchvision, Pillow, soundfile, librosa, decord, accelerate, or backend packages.
• Hub, network, cache, auth, gated model, or offline failures.
• device_map/device conflicts or placement warnings.
• dtype failures on CPU, MPS, unsupported GPU, or missing PyTorch.
• Unsupported task/model mismatches, wrong pipeline class, or missing processor/tokenizer files.
• Batch slowdowns, padding explosions, OOM, or list-vs-single output shape surprises.
• trust_remote_code prompts or unsafe custom-code requirements.

Minimal Examples

Text classification:

from transformers import pipeline

classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
result = classifier("Skill routing should be explicit.")
assert isinstance(result, list) and {"label", "score"} <= result[0].keys()

Image classification:

from transformers import pipeline

classifier = pipeline("image-classification", model="google/vit-base-patch16-224")
result = classifier(images="cat.png")
assert isinstance(result, list) and "score" in result[0]

ASR:

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="openai/whisper-large-v3")
result = asr(audio="clip.flac", return_timestamps=True)
assert "text" in result

Text generation with Pipeline:

from transformers import pipeline

generator = pipeline("text-generation", model="openai-community/gpt2", max_new_tokens=20)
result = generator("A compact smoke test", return_full_text=False)
assert "generated_text" in result[0]

For detailed chat/generation trade-offs, route to generation.

Evidence Base

This sub-skill distills Transformers 5.13.0.dev0 behavior from the README quickstart, quicktour, pipeline API docs, pipeline implementation, and pipeline tests. Runtime guidance is self-contained and does not require opening source files.