界面已本地化;技能介绍与说明文档按所选语言由 AI 自动翻译,代码、命令、路径和专业术语保留原文,以原作者文档为准。
探索技能
效率与协作 · SKILL.md

movement_scores

movement_scores

【效率与协作】Update movement-pruning importance scores from weight gradients and diagnose whether weights move away from zero.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准

技能说明

GitHub

name: movement_scores description: Update movement-pruning importance scores from weight gradients and diagnose whether weights move away from zero.

Straight-Through Movement Score Updates

Use this skill when a pruning or recovery harness needs the core first-order movement-pruning update. Do not use it as a generic magnitude pruning score; its purpose is to accumulate task fine-tuning movement.

Inputs

  • weights: numeric list or nested list.
  • scores: same shape as weights.
  • gradients: same shape, representing dL/dW.
  • lr_score: score learning rate.
  • Optional lr_weight for diagnostics of the associated SGD weight update.

Outputs

Updated scores and per-parameter diagnostics including score_delta, away_from_zero, and movement_label.

Workflow

  1. Flatten and shape-check weights, scores, and gradients.
  2. Compute the straight-through score gradient grad_s = grad_w * weight.
  3. Apply gradient descent score_new = score - lr_score * grad_s.
  4. Label weights as moving away from zero when an SGD step would increase absolute value.
  5. Return updated scores in the original shape.

Validation

Run python tests/test_movement_scores.py. Tests cover all sign combinations of weight and gradient and confirm scores increase exactly for away-from-zero movement.

Limitations

This skill does not compute model gradients; callers must supply gradients from their training objective.

我的收藏