界面已本地化;技能介绍与说明文档按所选语言由 AI 自动翻译,代码、命令、路径和专业术语保留原文,以原作者文档为准。
探索技能
效率与协作 · SKILL.md

coherence_recovery_evaluation

coherence_recovery_evaluation

【效率与协作】Evaluate coherence boosting recovery metrics and source-boundary mechanism checks for proxy experiments.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准

技能说明

GitHub

name: coherence_recovery_evaluation description: Evaluate coherence boosting recovery metrics and source-boundary mechanism checks for proxy experiments.

Coherence Recovery Evaluation

Use this skill after predictions have been produced by coherence boosting skills. It computes baseline and boosted metrics and records mechanism checks for reduced/proxy recovery.

Inputs

  • Labels, baseline predictions, boosted predictions, selected alpha, and invocation metadata.
  • A list of source paths used during recovery.

Outputs

  • Accuracy metrics and accuracy_gain_over_full_context.
  • Mechanism checks for full/short separation, alpha selection, no finetuning, and source-boundary compliance.

Workflow

  1. Compute exact accuracy for baseline and boosted predictions.
  2. Compute the gain over the full-context baseline.
  3. Verify no source path contains the original repository path marker.
  4. Emit booleans suitable for recovery_result.json.

Validation

Run tests/test_recovery_evaluation.py.

Limitations

This skill validates the recovery evidence; it does not claim full paper-scale reproduction unless the runtime evidence supports it.

我的收藏