---

name: layerwise_distillation_alignment description: Build dynamic teacher-student layer alignments and representation distillation losses for pruned transformer students.

layerwise_distillation_alignment

Use this skill when reconstructing the CoFi structured-pruning method or a mechanism-faithful reduced recovery. Do not use it as evidence for full GLUE/SQuAD reproduction unless it is embedded in a real pretrained-model training pipeline.

Inputs

• Small Python lists or JSON-compatible dictionaries describing model layers, masks, states, sparsity budgets, or proxy batches.
• For recovery, an attempt directory where logs and result files can be written.

Outputs

• Deterministic dictionaries containing mask composition, sparsity penalties, layer alignments, losses, parameter updates, or recovery traces.
• The scripts never read the original CoFi source repository.

Workflow

1. Import the script from scripts/ or invoke it through a recovery harness with the skill scripts directories on PYTHONPATH.
2. Validate all inputs are tiny, explicit, and JSON-compatible.
3. Execute the CoFi mechanism represented by this module and save any diagnostics needed by downstream recovery.
4. For proxy recovery, clearly mark the run as reduced and keep full-model training booleans false.

Validation

Run python /share/project/yuyang/workspace/Paper2Skills/Distiller/skills/module-to-skill/scripts/validate_skill_tree.py <skill_dir> --run-tests.

Limitations

This skill captures reusable CoFi mechanisms and deterministic tests. It is not a replacement for full BERT fine-tuning, GLUE/SQuAD preprocessing, or GPU latency benchmarking.