---

name: model-usage description: "Initialize OpenFlamingo models, load checkpoints, prepare multimodal prompts, run forward/generate, and debug media caching constraints." disable-model-invocation: true metadata: disco-role: operating license: MIT

OpenFlamingo Model Usage

Use this sub-skill when a task requires OpenFlamingo model construction, checkpoint loading, prompt/media preprocessing, generation, log-likelihood style forward() calls, media caching, or architecture debugging.

Do not start full model downloads, benchmark datasets, training, or evaluation unless the caller explicitly provides the required local files, network permission, hardware budget, and time budget. The safe default is to validate inputs and prepare code that works with a caller-supplied local cache or checkpoint.

Route by task

• Need API signatures or object relationships: read references/api-reference.md.
• Need an offline/local-cache generation recipe: read references/generation-workflows.md.
• Need to fix token, tensor, checkpoint, cache, decoder-layer, or import errors: read references/troubleshooting.md.
• Need a deterministic preflight check without downloads: run scripts/validate_generation_inputs.py.

Critical invariants

• Public imports are from open_flamingo import create_model_and_transforms, Flamingo.
• The package/distribution identity is open_flamingo 2.0.1.
• vision_x for Flamingo.forward() and Flamingo.generate() is B x T_img x F x C x H x W; current OpenFlamingo supports F=1 only.
• Text prompts must use the exact special tokens <image> and <|endofchunk|>. Each image/media item should have a matching <image> token in the prompt for the same example.
• For generation, set tokenizer.padding_side = "left" before tokenization and pass attention_mask.
• generate(num_beams=N) internally repeats vision_x along batch dimension when N > 1; do not pre-repeat the media batch for beam search.
• cache_media() is for repeated forward()/classification-style scoring, not for generate(). After cached calls, call uncache_media().

Safe input validation

From this sub-skill directory, validate a three-image captioning prompt without importing OpenFlamingo or downloading anything:

python scripts/validate_generation_inputs.py \
  --batch-size 1 \
  --num-media 3 \
  --num-frames 1 \
  --channels 3 \
  --height 224 \
  --width 224 \
  --prompt '<image>An image of two cats.<|endofchunk|><image>An image of a sink.<|endofchunk|><image>An image of'

If the script exits nonzero, correct the reported shape/token issue before attempting model execution.