executing-plans
executing-plans
Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available
スキルを探す
生産性・共同作業 · SKILL.md
off-policy-value
off-policy-valueGuides minimalRL DQN, ACER, and V-trace replay, target-network, and off-policy correction workflows for discrete control.
Independent GitHub snapshot · Automated classification · Original documentation governs
スキルの説明
GitHubname: off-policy-value description: "Guides minimalRL DQN, ACER, and V-trace replay, target-network, and off-policy correction workflows for discrete control." disable-model-invocation: true metadata: disco-role: operating license: MIT
Off-Policy Value Workflows
Use this sub-skill when a task asks about minimalRL's discrete value-learning, replay-buffer, target-network, ACER, or V-trace correction logic.
Route by task
- DQN / replay buffer / target network: read references/api-reference.md for
ReplayBuffer,Qnet, andtraincontracts, then references/workflows.md for the safe update flow. - ACER: read references/workflows.md for sequence replay and truncated importance sampling, then references/troubleshooting.md for probability-ratio failures.
- V-trace off-policy correction: use the V-trace section in references/api-reference.md and references/workflows.md; route single-process policy/value architecture questions to ../on-policy-discrete/SKILL.md.
- Shape-only or pre-training checks: run scripts/smoke_off_policy_value.py.
Do not use this sub-skill for
- REINFORCE, actor-critic, PPO, or recurrent PPO route selection; use ../on-policy-discrete/SKILL.md.
- DDPG, continuous PPO, or SAC; use ../continuous-control/SKILL.md.
- A2C/A3C multiprocessing and worker lifecycle; use ../parallel-actor-critic/SKILL.md.
Quick operating workflow
-
Identify whether the user is debugging value targets, replay sampling, behavior-policy probabilities, or target-network synchronization.
-
Check tuple contracts in references/api-reference.md before changing replay buffer contents.
-
Use references/workflows.md to preserve the update order: sample replay, compute detached targets, optimize current network, then update target/behavior state as appropriate.
-
Run the bundled smoke helper before long training:
python sub-skills/off-policy-value/scripts/smoke_off_policy_value.py --algorithm all -
For
ValueError: Sample larger than population, NaN ratios, gather shape failures, or stale target behavior, read references/troubleshooting.md.
Bundled runtime files
- references/api-reference.md records replay tuple contracts, classes, functions, hyperparameters, and tensor shapes.
- references/workflows.md explains DQN, ACER, and V-trace update sequences and adaptation rules.
- references/troubleshooting.md gives symptom-to-fix guidance for replay and correction failures.
- scripts/smoke_off_policy_value.py validates replay sampling and off-policy tensor contracts without importing the original repository or running full training.