---

name: paligemma-2 description: "Use Maestro's PaliGemma 2 recipe for JSON extraction, VQA, OCR-style fine-tuning, checkpoint loading, and inference." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

PaliGemma 2

Use this sub-skill when the user needs Maestro's paligemma_2 route for PaliGemma 2 text-generation fine-tuning or inference over images: JSON extraction, VQA, OCR-style prompts, loading/saving checkpoints, or choosing LoRA/QLoRA/freeze/full fine-tuning behavior.

Route here for

• CLI training through maestro paligemma_2 train.
• Python training through maestro.trainer.models.paligemma_2.core.PaliGemma2Configuration and train(config).
• Checkpoint work through load_model(...), save_model(...), and OptimizationStrategy.
• Single-image prediction through maestro.trainer.models.paligemma_2.inference.predict(...).
• PaliGemma-specific prompt handling, especially when and where Maestro prepends <image>.

Route elsewhere

• Generic JSONL split validation, Roboflow dataset identifiers, image-file checks, and metric registry details belong in datasets-and-metrics.
• Florence-2 or Qwen2.5-VL detection formatting belongs in their sibling sub-skills. Maestro's PaliGemma 2 implementation provides train/evaluation collate functions and text prediction; it does not expose a PaliGemma object-detection formatter API.
• Package-wide installation, root CLI discovery, and non-model-specific import warnings belong in the root installation and CLI guide and root troubleshooting reference.

Source-backed defaults to preserve

• Default model id: google/paligemma2-3b-pt-224.
• Default revision: refs/heads/main.
• Default train output base: ./training/paligemma_2; training creates a numbered run directory below it.
• Training/evaluation generation length default: PaliGemma2Configuration.max_new_tokens = 512.
• Direct prediction default: predict(..., max_new_tokens=1024).
• Dataset JSONL prefix values should not include <image> for standard Maestro use; train_collate_fn, evaluation_collate_fn, and predict prepend <image> internally.

Operating sequence

1. Confirm the dataset task and split layout with datasets-and-metrics before choosing model parameters.
2. Use API reference to choose CLI vs Python API, optimization strategy, checkpoint load/save behavior, and exact option names.
3. Use workflows for JSON extraction, VQA, OCR-style fine-tuning, inference, and safe validation steps.
4. Use troubleshooting for QLoRA/bitsandbytes, CUDA memory, invalid JSON suffixes, token truncation, HF access, and CLI edge cases.
5. Use the bundled helper only to generate configuration text; it never downloads models or starts training:

python scripts/build_paligemma_config.py \
  --dataset ./dataset \
  --optimization-strategy lora \
  --metric edit_distance \
  --metric bleu \
  --emit json