---

name: model-modules description: "Add and debug VLM-R1 model modules, including Qwen2VL, InternVL, GLM boundaries, processor inputs, freeze keywords, and custom reward hooks." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

VLM-R1 model modules

Use this sub-skill when a task is about adding, adapting, or debugging a VLM backend used by VLM-R1 GRPO training. This skill is intentionally router-like: use the references for the actual contract and failure details.

Route first

• Use this skill for:
  • adding a new module such as myvlm;
  • diagnosing model-name routing (qwen, internvl, glm);
  • checking processor/model input keys (pixel_values, image_grid_thw, image_flags);
  • choosing freeze-vision keywords;
  • understanding Qwen2VL, InternVL, and GLM support boundaries;
  • implementing module-level get_question_template or select_reward_func hooks.
• Route GRPO launch flags, DeepSpeed, LoRA command lines, multi-node, and W&B options to ../training-workflows/.
• Route generic JSONL schemas, global reward names, bbox scoring concepts, and non-module reward debugging to ../data-and-rewards/.
• Route saved REC/OVD evaluation output scoring to ../evaluation/.

Required workflow

1. Read references/model-modules.md for the module contract and backend-specific differences.

2. If editing or reviewing a module source file, run the no-import checker:

   python scripts/inspect_model_module_contract.py path/to/module.py --strict
   

   For a directory of module files, run:

   python scripts/inspect_model_module_contract.py path/to/vlm_modules --strict
   

3. For an added model backend, verify all of the following are deliberately handled before touching training launch commands:

   • subclass/interface methods;
   • model-name routing branch;
   • package exports/imports;
   • processor input keys and non-generate keys;
   • freeze-vision keywords;
   • prompt template for the requested task_type;
   • custom rewards only if is_reward_customized_from_vlm_module will be used.

4. If an error mentions GLM import or Glm4vForConditionalGeneration, use references/troubleshooting.md before assuming a broken checkpoint.

Fast decisions

• Qwen2-VL/Qwen2.5-VL: Qwen-style processor inputs with pixel_values and image_grid_thw; freeze keyword visual; pixel bounds are max_pixels and min_pixels.
• InternVL: remote-code AutoModel/AutoProcessor path; generation uses embeddings; processor/model inputs include pixel_values and image_flags; exclude image_flags from generate; freeze keyword vision_model; any-resolution cap is max_anyres_num.
• GLM: code exists, but the pinned Transformers environment used for this repository did not expose Glm4vForConditionalGeneration; treat GLM as unverified until imports and signatures are repaired.

Bundled files

• references/model-modules.md — complete distilled module contract, Qwen vs InternVL behavior, GLM caveats, and add-new-model checklist.
• references/troubleshooting.md — diagnosis recipes for routing, imports, processor key mismatches, freezing, pixel bounds, and custom rewards.
• scripts/inspect_model_module_contract.py — static AST checker for module files/directories; it never imports the repository.