---

name: learning description: "Configure REINVENT4 transfer learning and staged reinforcement learning runs safely." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

REINVENT4 Learning

Use this sub-skill when the user asks for transfer learning, transfer_learning.toml, staged RL/curriculum learning, staged_learning.toml, diversity filters, inception memory, TensorBoard monitoring, or a TL-then-RL workflow.

Route the request

• Use run_type = "transfer_learning" to bias a prior or checkpoint toward a curated SMILES set and write an output_model_file that can later be sampled or used as an RL agent_file.
• Use run_type = "staged_learning" for all reinforcement learning, including a single-stage RL run and multi-stage curriculum learning.
• For TL followed by RL, prepare and validate the TL config first, plan a brief sample/quality check of the TL model, then set the staged-learning agent_file to the chosen TL .model or checkpoint while keeping prior_file compatible with the same generator type.
• Cross-link to the scoring sub-skill for scoring component design, transforms, external scoring files, and scoring-only validation; this sub-skill only covers how scoring is embedded in staged learning.
• Cross-link to the sampling sub-skill when the user needs to sample from a prior, TL model, or RL checkpoint.
• Cross-link to the data-pipeline sub-skill when the training or seed SMILES file needs cleaning, splitting, column extraction, or standardization.

Safe workflow

1. Identify the generator family from the model file and input shape: Reinvent uses one-column SMILES; Mol2Mol/Pepinvent usually need conditional input plus sample_strategy; LibInvent/LinkInvent use two-column fragment inputs.
2. Use CPU by default for dry-run validation and small examples; if the config says device = "cuda:0", tell the user reinvent --device cpu FILE overrides the config for CPU fallback.
3. Validate config shape before any training: run python sub-skills/learning/scripts/check_learning_config.py CONFIG from the skill root or copy the script beside the planned config.
4. Never start long TL/RL training without explicit user approval for runtime, model files, input SMILES, scoring files, output locations, and device.
5. Prefer short exploratory settings first: small num_epochs for TL, conservative RL max_steps, and checkpoint files on every stage.

Required config anchors

• TL config: top-level run_type, device, optional top-level tb_logdir, then [parameters] with input_model_file, smiles_file, output_model_file, num_epochs, batch_size, and usually validation_smiles_file plus save_every_n_epochs.
• Mol2Mol TL: include [parameters.pairs] or dotted pairs.* thresholds for Tanimoto pair construction.
• RL config: top-level run_type, device, optional top-level tb_logdir, [parameters], [learning_strategy], optional [diversity_filter], optional [inception] or [intrinsic_penalty], and one or more [[stage]] blocks.
• RL parameters: prior_file, agent_file, summary_csv_prefix, batch_size, optional smiles_file for conditional generators, and safe checkpoint/continuation settings (use_checkpoint, purge_memories).
• RL stages: each [[stage]] needs max_steps, termination = "simple", min_steps, max_score, optional chkpt_file, and [stage.scoring] with inline components or filename plus filetype for bundled stage scoring files.

Bundled guidance

• Read references/tl-rl-workflows.md for concrete TL, staged RL, diversity filter, inception, intrinsic penalty, and TL-then-RL patterns.
• Read references/monitoring-and-outputs.md for TensorBoard log layout, CSV columns, checkpoint behavior, and review criteria.
• Read references/troubleshooting.md for device fallback, missing model/input files, model type mismatch, scoring-file path errors, SMILES column issues, TensorBoard confusion, remote responder setup, and long-run safeguards.
• Run scripts/check_learning_config.py to statically parse TOML/JSON/YAML configs and report missing major sections or risky settings without importing REINVENT4 or starting training.