---

name: model-api description: "Use when working with RobustVideoMatting MattingNetwork APIs, recurrent states, model variants, refiners, tensor shapes, and safe synthetic forward checks." disable-model-invocation: true metadata: disco-role: operating license: GPL 3.0

RobustVideoMatting Model API

Use this sub-skill when the task is about constructing or calling the RVM PyTorch model itself rather than converting videos, preparing training data, or running evaluation metrics.

Read this when

• The user asks how to instantiate MattingNetwork with mobilenetv3 or resnet50.
• The task mentions recurrent states, ConvGRU memory, downsample_ratio, segmentation_pass, foreground/alpha tensor shapes, or channel/rank errors.
• You need a safe import/forward smoke test before writing inference or training code.
• You need to decide whether to use deep_guided_filter or fast_guided_filter as the refiner.

Route other tasks elsewhere:

• Video/image conversion, checkpoints, TorchHub, TorchScript, ONNX, TensorFlow, or CoreML usage: inference-workflows.
• Dataset layouts, DATA_PATHS, training stages, losses, or augmentations: training-data.
• LR/HR metric evaluation or speed benchmarking: evaluation-tools.

Core workflow

1. Ensure the RVM source modules are importable as model and that PyTorch and TorchVision are installed. If in doubt, run the bundled smoke helper:

   python scripts/rvm_model_smoke.py --repo-root /path/to/RobustVideoMatting --variant mobilenetv3 --device cpu
   

2. Instantiate the model without downloading backbone weights unless the user explicitly wants that side effect:

   from model import MattingNetwork
   
   model = MattingNetwork(variant="mobilenetv3").eval()
   # or: MattingNetwork(variant="resnet50")
   

3. Feed RGB tensors normalized to 0..1 in channel-first format:

   • single frame/batch: [B, C, H, W]
   • chunked sequence: [B, T, C, H, W]

4. Recycle all four recurrent states in temporal order:

   rec = [None] * 4
   fgr, pha, *rec = model(src, *rec, downsample_ratio=0.25)
   

5. Validate the output contract. Matting mode returns foreground fgr, alpha pha, and four recurrent states. segmentation_pass=True returns segmentation logits plus recurrent states, not foreground/alpha.

Bundled references and scripts

• Read references/api-reference.md for verified constructor and forward signatures, tensor ranks, output shapes, recurrent state details, model variants, and architecture notes.
• Read references/troubleshooting.md when shape errors, invalid variants, device/dtype mismatches, recurrent-state misuse, pretrained downloads, or segmentation-pass confusion appear.
• Run scripts/rvm_model_smoke.py for a safe synthetic forward pass on CPU or CUDA. It is adapted from the repo's speed test but intentionally avoids benchmarks, downloads, and training.

Decision points

• Prefer mobilenetv3 for most inference and smoke tests because it is the smaller recommended variant. Use resnet50 when the user explicitly wants the larger model with modest quality improvement.
• Keep refiner="deep_guided_filter" unless the task specifically asks for the faster guided-filter refiner or export behavior.
• Use 5D input chunks ([B,T,C,H,W]) to improve parallelism while preserving temporal recurrence across chunks. Carry only the returned four state tensors into the next chunk.
• Do not use a CPU smoke test as proof of GPU speed, CUDA memory behavior, or high-resolution evaluation performance. Route those questions to evaluation-tools.

Acceptance check for model API answers

A good answer for this surface names the exact tensor rank/order, explains the four recurrent states, states whether the call returns fgr/pha or segmentation logits, includes a small validation snippet or smoke command, and routes any conversion/training/evaluation work to the owning sub-skill.