---

name: datasets description: "Loads TUM, ICL-NUIM, and ScanNet RGB-D sequences into deterministic batched tensors, with preprocessing, pose association, semantic labels, and RGBDImages handoff guidance." disable-model-invocation: true metadata: disco-role: operating license: MIT

Dataset adapters operating guide

Use this sub-skill when a task needs one of the repository's dataset adapters, its file/sequence conventions, RGB-D preprocessing, trajectory association, semantic labels, or a DataLoader batch ready for RGBDImages. The adapters consume data already present on disk. They do not download, extract, or repair datasets.

Keep the detailed contracts in the bundled references:

• API reference — constructors, utility functions, option gates, and output ordering.
• Workflows — preflight, sequence slicing, pose normalization, and DataLoader handoff.
• Data formats — directory trees, metadata line layouts, units, shapes, and label encodings.
• Troubleshooting — actionable diagnosis for missing files, malformed associations, empty sequences, and external-data limits.
• Layout checker — a deterministic, read-only checker for user-supplied dataset paths.

Route the request

1. Identify the adapter from the data contract, not from a directory name: TUM expects timestamp lists and optional quaternion ground truth, ICL expects TUM-compatible associations plus a block-matrix trajectory, and Scannet expects sequence-association metadata that points into scene extraction directories.
2. Confirm the external-data boundary before constructing the dataset. The package does not include these datasets, and the repository's TUM, ICL, and ScanNet native tests are data-gated. A missing local root is a precondition failure, not a reason to invent a substitute path or download data.
3. Run the layout checker against paths supplied by the caller. It only reads directory entries, metadata text, and selected path references; it never downloads, writes, renames, deletes, or mutates input data. A nonzero result means the adapter's expected structure is incomplete, not that the dataset itself is corrupt.
4. Choose seqlen, dilation, stride, start, end, output resolution, image layout, color normalization, and return flags deliberately. Record B, L, H, W, whether labels are requested, and whether poses are available before handing the batch to a structure.

Construct an adapter

Import directly from the dataset modules:

from gradslam.datasets.tum import TUM
from gradslam.datasets.icl import ICL
from gradslam.datasets.scannet import Scannet

The default return configuration asks for every output that the adapter can provide. That means TUM returns color, depth, intrinsics, normalized poses, relative transforms, names, and timestamps; ICL returns the same except for TUM timestamps; ScanNet returns those RGB-D/pose fields plus names and labels. Return flags remove fields from the tuple rather than inserting None values, so unpack only the fields enabled by the configuration. See the API reference for exact order.

Use a tuple for an explicit selection of sequences, trajectories, or scenes. A string is interpreted as a split-file path only when it exists; it is not a single sequence name. None means all discoverable items for TUM and ICL; for ScanNet it means all sequence metadata files. Lists are rejected by the adapter constructors.

For a first inspection, prefer a small seqlen and reduced height/width, keep shuffle=False, and request only fields needed by the next operation. DataLoader is ordinary PyTorch collation: with batch_size=B, each image field gains a leading batch dimension and names/timestamps are collated as a sequence of strings (commonly a list).

Preserve preprocessing contracts

• Color is resized with bilinear interpolation. By default it remains in its source-like [0, 255] scale; normalize_color=True converts it to approximately [0, 1].
• Depth is resized with nearest-neighbor interpolation, gains a singleton channel, and is divided by the dataset scale: 5000.0 for TUM and ICL, 1000.0 for ScanNet. The resulting depth is in the adapter's meter-like convention.
• channels_first=False yields (L,H,W,3) color and (L,H,W,1) depth; channels_first=True yields (L,3,H,W) and (L,1,H,W). A collated batch therefore has (B,L,...). Intrinsics and poses remain (B,1,4,4) and (B,L,4,4).
• Intrinsics are scaled independently by height/480 and width/640. Scale fx and cx by the width ratio, and fy and cy by the height ratio. Do not rescale a matrix twice.
• Pose and transform fields are only loaded when return_pose or return_transform requires them. Each sequence is normalized to the first pose; its first normalized pose and first relative transform are identity.

Handoff into RGBDImages

For an ordinary channels-last batch with all required fields:

from torch.utils.data import DataLoader
from gradslam.structures.rgbdimages import RGBDImages

loader = DataLoader(dataset, batch_size=2, shuffle=False)
colors, depths, intrinsics, poses, transforms, names = next(iter(loader))[:6]
rgbd = RGBDImages(colors, depths, intrinsics, poses, channels_first=False)

The exact slice above is appropriate for ICL; TUM has a seventh timestamp field and ScanNet has a seventh label field. The transforms, names, timestamps, and labels are not constructor arguments for RGBDImages; retain them separately. If channels_first=True, pass that same flag to RGBDImages. Do not silently omit intrinsics or replace absent poses with an identity tensor: a caller that needs geometry must request/provide them.

For ScanNet labels, keep the label tensor separate from RGB-D geometry. Labels are resized with nearest-neighbor interpolation and returned with a singleton last channel by the implementation, so a collated batch is typically (B,L,H,W,1) even when channels_first=True; older prose may describe it as (L,H,W). Use seg_classes="nyu40" to retain the source indexing, or seg_classes="scannet20" to remap the supported source ids to the contiguous 20-class palette. Use get_color_encoding and datautils.create_label_image only when a visualization or colorized label image is explicitly needed, and verify the palette representation against the installed release.

Boundaries and verification

This skill does not promise that any external dataset is present, that a split file names valid items, or that all referenced image/pose files are readable. The checker reports path existence only; adapter construction and one small __getitem__ are the next data-dependent verification step. Do not run network downloads, full native dataset tests, GPU flows, GUI visualization, or notebook execution while drafting or when only a layout check was requested.

The deterministic CPU utility coverage is the strongest data-independent signal. TUM, ICL, and ScanNet adapter tests/examples are not guaranteed without their external data and prepared extraction metadata. Keep that status explicit in reports and handoffs.