---

name: variable-shapes-video description: "Routes vit-pytorch variable-resolution NaViT batches, nested tensor variants, 1D/3D/N-D tensors, and video transformer workflows." disable-model-invocation: true metadata: disco-role: operating license: MIT

Variable Shapes and Video

Use this sub-skill when a user needs vit-pytorch guidance for variable spatial sizes, non-image tensor ranks, or video inputs. Route prompts here when they mention NaViT, na_vit_nested_tensor, na_vit_nested_tensor_3d, vit_3d, simple_vit_3d, cct_3d, vivit, vivit_with_moss, vit_1d, simple_vit_1d, vit_nd, vit_nd_pope, vit_nd_rotary, accept_video_wrapper, variable-resolution grouping, max token budgets, frame patching, or N-D patch grids.

Route by user intent

• Variable-resolution images in one forward: use vit_pytorch.na_vit.NaViT. Read workflows before choosing manual List[List[Tensor]] grouping or group_images=True auto grouping with group_max_seq_len.
• Nested tensor NaViT: use vit_pytorch.na_vit_nested_tensor.NaViT for a flat list of image tensors, or vit_pytorch.na_vit_nested_tensor_3d.NaViT for a flat list of (channels, frames, height, width) volumes, only after checking PyTorch nested/jagged support. Read nested tensor caveats and troubleshooting.
• 3D and video classifiers: use vit_pytorch.vit_3d.ViT, vit_pytorch.simple_vit_3d.SimpleViT, vit_pytorch.cct_3d.CCT, vit_pytorch.vivit.ViViT, or vit_pytorch.vivit_with_moss.ViViT for inputs shaped (batch, channels, frames, height, width). Read video layouts before setting frames, frame_patch_size, and image_patch_size.
• Video wrapper around an image model: use vit_pytorch.accept_video_wrapper.AcceptVideoWrapper when the user already has an image backbone and wants to run it per frame, optionally restoring time as an output dimension, adding time positional embeddings, or applying MOSS over patch-token outputs.
• 1D or N-D tensors: use vit_pytorch.vit_1d.ViT, vit_pytorch.simple_vit_1d.SimpleViT, vit_pytorch.vit_nd.ViTND, vit_pytorch.vit_nd_pope.ViTND, or vit_pytorch.vit_nd_rotary.ViTND. Read 1D and N-D routing for input layouts, tuple length rules, and patch-grid outputs.

Boundaries

This sub-skill owns variable-resolution image grouping, max sequence length packing, patch and grid position handling, video frame/spatial patch shapes, N-D patching, and nested-tensor caveats.

Route elsewhere instead of duplicating guidance:

• Pure image-only architecture selection, standard square/non-square image classifier choice, and image architecture comparisons belong to the image-architecture route unless the prompt asks about variable sizes or non-image ranks.
• Loss-based pretraining, distillation, masked modeling, DINO/EsViT, MAE/SimMIM/MPP, or adaptation wrappers belong to the pretraining/adaptation route.
• Attention inspection-only wrappers such as Recorder and Extractor belong to the introspection/customization route unless they are only incidental to a variable-shape or video workflow.

Operating rules for future agents

1. Normalize the tensor layout before constructing a model. NaViT images are unbatched (channels, height, width) tensors in Python lists; video/3D models consume (batch, channels, frames, height, width); 1D models consume (batch, channels, seq_len); N-D models consume (batch, channels, *input_shape).

2. Check divisibility before running. Height and width must be divisible by patch_size or image_patch_size; frames must be divisible by frame_patch_size; each N-D dimension must be divisible by its corresponding patch dimension. See troubleshooting.

3. For NaViT, compute token counts before grouping. A standard NaViT image contributes (height // patch_size) * (width // patch_size) tokens before optional token dropout. group_images=True greedily preserves input order and starts a new group when adding the next image would exceed group_max_seq_len; it cannot rescue one image whose token count is already too large.

4. Separate temporal and spatial patch sizes. frame_patch_size patches along the frame axis; image_patch_size patches height and width. Do not swap them, and do not use (batch, frames, channels, height, width) with these modules.

5. Treat nested tensor paths as version-sensitive. Prefer standard na_vit.NaViT for broadly portable examples. Use nested tensor variants only after a small runtime probe confirms the installed PyTorch supports the required nested/jagged operations.

6. Use the bundled smoke helper for confidence. After installing vit-pytorch, run:

   python sub-skills/variable-shapes-video/scripts/smoke_variable_shapes_video.py
   

   The helper imports the installed package, uses tiny CPU tensors, and does not read repository source files.

References

• Grouping rules, layouts, and supported families
• Shape, grouping, version, and wrapper troubleshooting
• CPU smoke helper