---

name: model-architecture description: "Inspect and configure HRM ACT v1 model internals, losses, FlashAttention layers, and dynamic model identifiers." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

HRM Model Architecture

Use this sub-skill when the task is to understand, configure, inspect, or debug the Hierarchical Reasoning Model (HRM) ACT v1 implementation rather than to run full training.

When to use

• The user asks about HRM high-level/low-level recurrent modules, ACT halting, Q heads, carry state, sparse puzzle embeddings, StableMax loss, or FlashAttention-backed attention blocks.
• A config mentions arch.name=hrm.hrm_act_v1@HierarchicalReasoningModel_ACTV1 or arch.loss.name=losses@ACTLossHead.
• A model import fails because flash_attn, flash_attn_interface, adam_atan2_backend, CUDA, or dynamic module@class identifiers are wrong.
• The user needs to alter hidden size, heads, cycles, layers, positional encodings, halt steps, or loss type safely.

Route map

1. Read references/architecture.md for the HRM ACT v1 object graph, carry objects, input/output tensors, and halting mechanics.
2. Read references/api-reference.md for verified config fields, function/class signatures, losses, and dynamic identifier rules.
3. Read references/troubleshooting.md for import/backend/config errors.
4. Run scripts/inspect_model_config.py to inspect identifiers and optionally run CUDA dependency smokes from an HRM checkout.

Critical facts

• The default architecture config uses hrm.hrm_act_v1@HierarchicalReasoningModel_ACTV1 wrapped by losses@ACTLossHead with loss_type=stablemax_cross_entropy.
• models/layers.py imports FlashAttention at module import time. If neither flash_attn_interface nor flash_attn is importable, HRM model imports fail.
• pretrain.py imports AdamATan2 from adam_atan2; real training requires the compiled adam_atan2_backend CUDA extension.
• Real model construction and forward execution are CUDA-oriented. CPU imports and source inspection are useful for config facts but do not verify the required training/evaluation backend.
• The repository's tiny model smoke under current dependency versions exposed a runtime issue where FlashAttention output may be non-contiguous and models/layers.py calls .view(...); use this as troubleshooting evidence if a forward pass fails with a stride/view error.

Boundaries

• This sub-skill owns model/loss/config/API understanding and environment smokes for model dependencies.
• Use data-preparation for dataset schema, builders, and visualization.
• Use training-evaluation for Hydra training commands, checkpoint evaluation, W&B, and ARC post-processing.
• Do not present a CPU import as proof that HRM can train or evaluate. The backend gate is CUDA + FlashAttention + adam-atan2 backend.

Bounded inspection

From an HRM checkout, run:

python <skill>/sub-skills/model-architecture/scripts/inspect_model_config.py \
  --repo-root /path/to/HRM --json

If a CUDA environment is prepared and the user wants dependency readiness:

python <skill>/sub-skills/model-architecture/scripts/inspect_model_config.py \
  --repo-root /path/to/HRM --cuda-smoke

The helper verifies imports and dependency smokes. It intentionally avoids long training and does not require real datasets or checkpoints.