---

name: paper-skills-distiller description: "Orchestrate a paper-to-skills distillation loop for AI research papers supplied as local files, URLs, arXiv ids, or titles, optionally with a local or remote repository, by resolving sources, modularizing the paper, creating module skills, preparing recovery runtime, recovering a paper result without the source repo, analyzing gaps, and iterating until acceptable." metadata: disco-role: meta

Paper Skills Distiller

Use this skill as the top-level controller when a user provides a research paper source, with or without an implementation repository source, and wants the paper converted into reusable Agent Skills. The paper source may already be local, or it may be a direct PDF URL, arXiv URL/id, or paper title that must be resolved before the run starts.

All generated skill content must be written in English. Keep user-facing status in the user's language if appropriate, but artifact names, schemas, SKILL.md files, scripts, and validation reports should be English.

The default refinement budget is 10 refine cycles after the first recovery. Use the user's supplied iteration_budget when present; otherwise record iteration_budget: 10 in run_manifest.json. The user may set any non-negative integer. Stop before starting a refine cycle that would exceed this budget, and summarize the remaining feedback instead of continuing silently.

The default recovery mode is hard. In hard mode, the run must not accept reduced, proxy, toy, smaller-model, or fallback recovery as success; if the full requested standard cannot run, record a concrete blocker or return refine. In soft mode, a reduced or proxy recovery may be accepted only when it is explicitly declared, justified, executable, and passes the recovery experiment gate.

Prefer a TOML run config when the user provides one. Use ../create-paper-skills/assets/distiller-run-config-template.toml as the fill-in template and validate it with scripts/read_run_config.py before resolving sources or initializing runs. A config may contain one or more [[runs]] entries for batch distillation.

Default outputs use one run directory with two main children: <workspace_root>/<paper_slug>/distillation/ for process artifacts and <workspace_root>/<paper_slug>/skills/ for generated reusable skills. Do not create numbered attempt directories unless the user explicitly supplies a custom attempt_dir for that purpose.

Distillation Model

Use the paper or tutorial source as anchor z for task-agnostic distillation unless the user supplies a concrete downstream replication task. For a concrete task, record the task anchor as tau = (q, D, E, g) and use task-oriented distillation; do not create a synthetic task tuple for a paper-only run.

The run follows four stages:

1. Scope the required capabilities Q, module boundaries, non-goals, and verification targets.
2. Ground Q in paper and permitted source evidence X, recording provenance, revisions, exclusions, conflicts, and assumptions.
3. Construct candidate graph G_tilde from module skills, references, scripts, and links.
4. Verify candidate G_tilde into accepted graph G and construction record R through executable recovery, validation, refinement, deployment review, and Researcher handoff. Task-agnostic runs use representative operating checks when no downstream task-level outcome is defined.

Source Rules

• Paper: mandatory. Read the full paper or a faithful extracted text version. If the user gives a URL, arXiv id, or title instead of a local path, first resolve it to a local PDF/text artifact and save source acquisition evidence.
• Repository: optional. It may be a local path, Git URL, none, or unknown. Use it only during modularization and module skill creation. Treat it as implementation evidence, not as a substitute for paper understanding. If it is unknown, follow repo_discovery_mode: ask records candidates/blockers and asks before searching or selecting, auto may run bounded GitHub discovery and clone the top candidate, and disabled proceeds paper-only.
• Recovery: must not read the original source repository. During recovery, use only the paper, module documents, generated module skills, data, and general package documentation.
• Internet: use only when needed for source acquisition, repo discovery, datasets, package docs, or web-search experiments. If the environment provides a network setup script, source it before network commands. Keep downloads, title searches, repo discovery, and clones bounded and logged.
• Logs: every run must be recorded under its configured distillation directory.

Required Loop

1. If a run config path is supplied, validate and normalize it with scripts/read_run_config.py; otherwise collect the same fields from the user's prompt. For batch configs, process each normalized run independently and keep separate source-resolution, attempt, and summary artifacts. Use read_run_config.py --slug <paper_slug> --run-only to write the selected normalized run JSON passed to init_run.py --run-config.
2. Resolve paper and optional repo sources when needed with scripts/resolve_sources.py or an equivalent bounded, logged procedure. The resolver covers local paths, direct URLs, arXiv URLs/ids, arXiv title search, Git repo URLs, and repo_discovery_mode=auto GitHub repo discovery. If a title is not found there, use web search for official or scholarly sources and record the chosen URL. If title or repo discovery is ambiguous, ask the user to choose before proceeding.
3. Initialize the distillation directory with scripts/init_run.py using the resolved local paper path, optional resolved repo path, configured iteration_budget, configured recovery_mode, configured generated_skills_root, optional exact attempt_dir, and --run-config when a normalized config JSON exists.
4. Load plan-paper-skill-modules and create:
   • paper_profile.md
   • module_plan.json
   • one module document per module in modules/
   • no more than five modules
5. Validate the module plan with plan-paper-skill-modules/scripts/validate_module_plan.py.
6. For each module, load create-paper-module-skill and create the module skill under the configured output skill root.
7. Validate every generated module skill with create-paper-module-skill/scripts/validate_skill_tree.py and run its tests or smoke checks.
8. Load prepare-paper-recovery-env and create environment/runtime_handoff.json from bounded package, model, GPU, benchmark, and dataset preparation. If required packages, datasets, or model caches are missing, run the active preparation path first: create/reuse an isolated recovery environment under $DISCO_CODING_AGENT_DIR/envs/ or ~/.disco/agent/envs/, install the smallest missing package set, and attempt permitted bounded model/data acquisition before treating the item as blocked.
9. Load recover-paper-result and reproduce one fast, meaningful paper result without reading the original repo, using the runtime handoff as recovery input.
10. Load analyze-paper-recovery and compare the recovery result with the paper target or an explicitly declared proxy target.
11. If analysis says refine, update the relevant workflow guidance, module documents, module skills, environment handoff usage, and recovery harness, then rerun steps 7-10. Stop when status is accept or the configured refinement budget is exhausted.
12. After an accepted run passes final and reported validation, classify the complete generated module graph for deployment. Default task-, project-, private-data-, evaluator-, benchmark-, or environment-bound output and all uncertain cases to project scope at <project-dir>/.agents/skills/. Use managed scope at ~/.disco/agent/skills/ only with evidence that the graph is self-contained, provenance-backed, independent of transient run state, representative-use verified, and reusable across projects or research tasks. Keep every module in the same scope.
13. Present the classification evidence, exact targets, entry point, verification, unresolved limits, collisions, and shadowing impact. After approval, use the visible distill-ml-knowledge/scripts/import_operating_skill_graph.mjs once with every top-level module root and the selected scope. Overwrite requires separate approval. Write researcher-handoff.md with the selected scope and exact imported paths, then suggest /researcher; if import is declined, record that the graph remains staged.

Ordering and Runtime Discipline

• Finish and validate the modularization artifacts before heavyweight runtime probing. Do not delay paper_profile.md, module_plan.json, or modules/*.md while searching for models, package caches, datasets, or GPUs.
• Source acquisition happens before distillation initialization when needed, but it should stay bounded: direct local path checks first, then direct URL/arXiv download, then title search only if requested by the user's input, then repo discovery only when repo_discovery_mode or user confirmation permits it. Do not treat source acquisition as recovery evidence for reading the original repo.
• Keep filesystem discovery bounded. Do not run broad scans such as find /share/project ... or recursive searches across NFS roots. Search only the workspace, user-supplied paths, known cache roots, or a short explicit allowlist, and use timeouts for cache probes.
• Long package installs, model downloads, and benchmark setup commands must have a time budget. If they stall, stop them, record the exact blocker in recovery artifacts, and continue with the best valid lower-cost recovery allowed by the user.
• Missing local dependencies, model caches, datasets, or search credentials are not by themselves sufficient to end the run as blocked. First attempt the allowed setup path: isolated environment creation, targeted package installation, bounded dataset/model acquisition, configured VPN/proxy or mirror use, and keyless/public alternatives where valid. Only mark blocked after those actions are logged or after asking the user for required credentials/permissions that cannot be inferred.
• A fallback or proxy can be useful evidence, but it must not be counted as a successful required runtime unless the recorded mechanism checks prove that the required data item, model/runtime, loss, and optimizer or training step actually ran.
• In hard recovery mode, do not count fallback, proxy, reduced training, subset-only evaluation, or smaller-model execution as accepted recovery. Such artifacts may be logged as diagnostics, but analysis must return refine or blocked unless the full requested standard ran.
• In soft recovery mode, reduced or proxy recovery is allowed only as a declared target with executable evidence, generated-skill invocation logs, and recovery/experiment_validation.json passing.
• Independent runs must not silently depend on previous runs. Reuse immutable local datasets or benchmark checkouts only as recorded benchmark sources when fresh acquisition is blocked or the user supplied them. Do not reuse previous generated skills, data items, prompts, outputs, training logs, or recovery results as evidence for a new run.
• Runtime setup must produce a handoff before recovery. recover-paper-result should not repeat broad environment discovery or silently invent fallback state; it should consume environment/runtime_handoff.json and add that handoff to recovery/source_manifest.json.

Distillation Directory Contract

Each distillation directory should contain:

run_manifest.json
run_config.normalized.json   # optional but preferred when a config was used
source_resolution.json       # optional, when paper/repo source acquisition was needed
paper_profile.md
module_plan.json
modules/
  <module_id>.md
generated_skills_validation/
  <module_id>.json
environment/
  runtime_handoff.json
  logs/
    command_log.json
recovery/
  experiment_plan.md
  experiment_validation.json
  source_manifest.json
  recovery_result.json
  logs/
    experiment_command_log.json
    generated_skill_invocations.json
analysis/
  analysis_report.json
  feedback.md
final_validation.json
reports/
  generated-skills/
  verification/
  final/
    final_report.json
    final_report.md

Use additional files when useful, but do not omit the core contract. Record failures as artifacts instead of relying on terminal output.

Quality Bar

A generated module skill is acceptable only if it has:

• A clear trigger description in frontmatter.
• A concise workflow that states when to use the skill and when not to use it.
• Explicit input/output contracts.
• At least one deterministic test, smoke script, or validation command.
• A saved validation JSON whose ok is true and whose tests.attempted is true. Prose-only inspection is not enough for either hard or soft mode.
• No dependence on the original paper repository unless the skill's purpose is explicitly to operate on that repository.
• Enough implementation detail for a future agent run to use the skill without rereading the entire paper.

The whole paper distillation is acceptable only if:

• The module split covers the paper's main method and evaluation pathway.
• environment/runtime_handoff.json records package availability, GPU status, model readiness or blockers, benchmark acquisition/reuse, environment mutation status, and allowed recovery sources.
• environment/logs/command_log.json records subprocess commands run during environment preparation, including timed-out or failed commands.
• The recovery run exercises the generated skills, not a hand-written one-off solution.
• The recovery result is produced by an executable experiment command whose command log is saved under recovery/logs/experiment_command_log.json.
• recovery/experiment_plan.md explains full/reduced/toy options and why the chosen target is the strongest feasible experiment.
• recovery/logs/generated_skill_invocations.json proves generated module skills were called, imported, or cross-checked.
• The recovery experiment gate is run and saved to recovery/experiment_validation.json with ok: true.
• The recovery run exercises every generated core module that the proxy claims to validate, or explicitly records why a module is not applicable. If a generated scoring/loss module exists separately from the training harness, the recovery cross-checks it rather than relying only on duplicated harness logic.
• Proxy recovery records explicit mechanism_checks that demonstrate the core paper mechanism ran. Do not accept a proxy solely because EM/F1 or another task metric is high.
• If recovery uses a benchmark or dataset repository, the item log and source manifest identify concrete resource files used to derive the sample whenever feasible. A benchmark-style item without resource-file provenance must be labeled as such and justified.
• If recovery reuses a local benchmark checkout from another run or workspace location, the source manifest records the fresh-fetch blocker, reused path, commit/version, concrete files used, and a snapshot or copy of those files inside the current distillation directory.
• At least one recovery run uses the strongest practical real runtime available in the workspace when the user provides one. If the user gives a model path, GPU ids, or an inference environment, record whether it was used; if it could not be used, record the exact blocker and run the best lower-cost fallback.
• The analysis report explains metric gaps and either accepts the result or gives actionable module-level feedback.
• The final artifact contract validator is run and its JSON output is saved to final_validation.json.
• The organized final report is written to reports/final/final_report.md and reports/final/final_report.json, and the reported validation stage passes.
• The complete module graph has a recorded reusability classification. Any live deployment uses one approved scope, excludes process/review artifacts, and is recorded with exact paths in researcher-handoff.md.
• run_manifest.json and run_config.normalized.json record recovery_mode, and hard-mode runs are not accepted if recovery_result.json marks is_proxy: true or mechanism_checks.reduced_training_executed: true.
• When paper/repo source acquisition was needed, the resolved local paths, commands/network operations, selected title/arXiv id, and blockers are recorded in source/source_resolution.json or reports/final/final_report.md.

Refinement Rules From Real Runs

• Do not accept a semantically correct but metric-failing result without improving the relevant evaluation or answer-formatting skill.
• If an intermediate module output contaminates later stages, refine that module's output contract and add a regression test before rerunning recovery.
• If a runtime environment is incompatible, do not mutate it unless the user explicitly allows it. Prefer an existing compatible environment or create a separate environment under $DISCO_CODING_AGENT_DIR/envs/ or ~/.disco/agent/envs/, then record the choice in source_manifest.json.
• After any refinement, rerun module skill validation and recovery in the current distillation directory or a clearly versioned cycle subdirectory.

Practical Guidance

Favor small, composable module skills over one monolithic "paper skill". A good split usually separates protocol/data formatting, retrieval/tool adapters, core algorithm loop, scoring/evaluation, and experiment recovery. Keep the number of modules below five by merging thin utility modules into the nearest substantive module.

If the full paper result requires heavy training or unavailable models, check recovery_mode before lowering the target. In hard mode, record the blocker and do not accept a proxy as success. In soft mode, recover a declared proxy result that preserves the paper's mechanism. The proxy must identify the dataset, sample size, metric, paper target, and why the proxy is the fastest faithful test.

Scripts

• scripts/read_run_config.py: validate and normalize a TOML config with one or more Distiller runs.
• scripts/resolve_sources.py: resolve local or remote paper/repo sources into local paths before distillation initialization, including optional bounded repo discovery.
• scripts/init_run.py: create the distillation directory and manifest, optionally copying a selected normalized run config into the run.
• scripts/validate_distillation_run.py: validate that required artifacts exist at each stage.
• scripts/write_final_report.py: write reports/final/final_report.json and reports/final/final_report.md from the distillation artifacts.