---

name: pipeline description: "HyperTools stage pipelines, model-spec grammar, and return-model reuse." disable-model-invocation: true metadata: disco-role: operating license: MIT

HyperTools Pipeline

Use this sub-skill when the task is about the HyperTools analysis pipeline: manip, normalize, reduce, align, cluster, apply_model, Pipeline, or reusing a fitted result on held-out data.

Route here when the user wants

• A staged analysis chain with canonical ordering.
• A reusable fitted model returned by return_model=True.
• A Pipeline object or a pipeline= reuse path.
• Model-spec grammar help: strings, dicts, classes, instances, lists.
• ndims / random_state behavior for reduction or clustering.
• Cluster labels, mixture proportions, or label recovery from a fitted chain.

Route elsewhere when the user wants

• Plot styling, rendering, backends, hues, trails, animation, or surfaces: use ../visualization/.
• Text vectorizers, semantic models, or corpus choices: use ../text/.
• Forecasting or dedicated imputation models: use ../forecasting/.
  • Only impute= at format time is in scope here.

Read first

• references/pipeline-reference.md for signatures, grammars, registries, canonical order, and reuse rules.
• references/workflows.md for fit-once / reuse recipes and common chains.
• references/troubleshooting.md for unknown-model, ndims, shape, and reuse failures.
• scripts/smoke_pipeline.py for a tiny CPU end-to-end smoke check.

Quick decision guide

1. Need to apply one model family to stacked data?
   • Use apply_model.
2. Need a stage-wise analysis chain in HyperTools order?
   • Use analyze, reduce, align, normalize, manip, or cluster.
3. Need to reuse the exact fit on new data?
   • Pass the fitted wrapper back as the model spec, or pass a fitted Pipeline to analyze(..., pipeline=...).
4. Need to compose a named chain directly?
   • Use Pipeline(steps) or the dispatcher kwargs that return one.

Canonical order

HyperTools stages always follow the same order when dispatcher kwargs are used:

manip -> normalize -> reduce -> align -> cluster

A few important corollaries:

• Missing-data fill happens before these stages, during format time.
• reduce happens before align in the canonical chain.
• cluster labels the final geometry, so analyze(..., cluster=...) returns transformed data, not labels.
• apply_model is separate from the stage pipeline and does not include manip, align, or impute.

Return-model rule of thumb

• Single stage: return_model=True yields that stage's fitted wrapper.
• Multiple stages or cross-module kwargs: return_model=True yields a fitted Pipeline.
• Reuse the fitted wrapper or Pipeline on new data instead of refitting.

What to remember

• Model specs can be strings, dicts, classes, instances, or lists.
• Pipeline stores already-resolved steps and replays them in order.
• Pipeline.transform reuses fitted steps; some non-transformable reducers or hard clusterers still have limits on held-out data.
• analyze(..., pipeline=...) is the replay path for a fitted chain.
• When labels are needed, recover them from the fitted cluster step.

If the task starts drifting into visualization or text/forecasting details, hand it off to the sibling sub-skill and keep this one focused on stage dispatch, model grammar, and reuse.