---

name: structures description: "This sub-skill guides construction, conversion, transformation, and safe inspection of gradslam RGB-D and batched point-cloud structures." disable-model-invocation: true metadata: disco-role: operating license: MIT

Structures operating guide

Use this sub-skill when a task must build or manipulate RGBDImages, build a Pointclouds object, convert RGB-D frames to points, preserve tensor layout, or prepare a non-interactive Open3D/Plotly representation. Keep the complete API details in the linked references:

• API reference for signatures, properties, and mutation semantics.
• Data formats for shape, layout, depth, camera, and ragged-batch contracts.
• Workflows for construction, conversion, and device-safe recipes.
• Troubleshooting for failures and adapter limits.
• Tiny smoke helper for a deterministic, external-data-free check.

Route the request

1. Identify whether the input is a sequence of RGB-D frames or an existing point cloud. Record B, L, H, W, representation (channels_first or channels-last), device, depth units, and whether poses are available.
2. Normalize image tensors before construction. The RGB-D constructors accept PyTorch tensors only; NumPy conversion belongs at an input boundary. RGB is not automatically divided by 255 and depth is not rescaled.
3. Choose one representation deliberately. Use channels-last for ordinary image inspection, Plotly RGB-D animation, and point-cloud conversion. Use channels-first only when the consumer explicitly requires it.
4. Keep all constructor tensors on one device. Do not rely on an implicit cross-device copy to repair mixed inputs. Verify the selected destination after construction.
5. For visual output, prefer as_figure=False first so an adapter can be checked without opening a browser or GUI. Call .show() or Open3D drawing only in an explicitly interactive environment.
6. If the request is an RGB-D-to-point-cloud conversion, select a one-frame object (rgbd[:, frame]) before calling the helper; it requires L == 1. Choose filtered ragged output or dense padded output deliberately.
7. Follow workflows for the concrete recipe and troubleshooting for constructor, cache, representation, device, conversion, and adapter errors before expanding a fixture or invoking external data.

Build RGBDImages

The constructor is:

RGBDImages(rgb_image, depth_image, intrinsics, poses=None,
           channels_first=False, device=None, *, pixel_pos=None)

Supply five-dimensional RGB and depth tensors, a (B, 1, 4, 4) intrinsics batch, and optionally (B, L, 4, 4) poses. Channels-last means rgb=(B,L,H,W,3) and depth=(B,L,H,W,1); channels-first means rgb=(B,L,3,H,W) and depth=(B,L,1,H,W). The object reports shape == (B,L,H,W), exposes cdim as 4 or 2, and preserves the chosen layout. A missing pose is valid: poses is None, has_poses is false, and global maps fall back to copies of local maps.

After construction, inspect rgb_image, depth_image, intrinsics, poses, valid_depth_mask, and has_poses before deriving geometry. Accessing vertex_map, normal_map, global_vertex_map, or global_normal_map computes and caches the result. Zero or negative depth is masked to zero in vertex maps; normal maps are likewise zero at missing-depth pixels. Derived maps are in the same channel layout as the object.

Use to_channels_last(copy=False) or to_channels_first(copy=False) for an out-of-place conversion. These return the same object when already in the requested layout unless copy=True; the underscore forms mutate in place. Index only batch and sequence dimensions. Integer indexing retains singleton batch/sequence dimensions, which is useful for downstream conversion.

Build and manipulate Pointclouds

Construct from either a list of ragged (N_b,3) tensors or an equally padded (B,N,3) tensor. Optional normals and colors match point shapes; optional features use (N_b,C) or (B,N,C) and share C across the batch. Attributes must use the same list-versus-tensor representation as points. Pointclouds() is a supported empty object; an empty list of points is not.

Use points_list and points_padded according to the consumer. List-backed inputs are padded lazily with zero rows, and nonpad_mask plus num_points_per_pointcloud identify real rows. The reverse list properties remove padded rows. has_normals, has_colors, and has_features are the presence checks; absent attributes remain None, not synthesized zeros.

For safe transformations, prefer out-of-place +, -, *, /, rotate, transform, or pinhole_projection. Use the underscore variants only when mutation is intended. Offset and scale affect points only; rotation and rigid transformation affect points and normals; colors and features are unchanged. Rotations accept (3,3) or (B,3,3), transformations (4,4) or (B,4,4), and the batch dimension must match. pinhole_projection accepts (4,4) or (B,4,4) intrinsics and places projected points on the z=1 plane.

Convert RGB-D to point clouds

Call pointclouds_from_rgbdimages(rgbd, global_coordinates=True, filter_missing_depths=True) only for an RGBDImages object with L == 1. The conversion selects global or local vertex/normal maps, converts to channels-last, and returns points, normals, and RGB colors. Filtering removes pixels whose depth is not positive; disabling it preserves a dense flattened cloud including zero-filled missing-depth rows. If poses are absent, global coordinates intentionally equal local coordinates.

Copy, gradients, and devices

clone() makes independent tensor storage. detach() makes a cloned object whose internal tensors do not require gradients. to(device, copy=False), .cpu(), and .cuda() transfer internal tensors; copy=True forces a new object when supported. Use torch.device(...) when comparing devices and ensure operands of transforms and append_points are on the same device.

Property setters preserve the existing batch/point counts and validate the relevant shape. Padded setters also require the established device and zero padding; list setters clone values onto the object's device. Setting a list or padded representation invalidates the alternate representation in the intended API. Avoid mutating a returned list tensor in place when the padded cache may already exist; re-check both representations after structural edits.

Adapters and verification

• Pointclouds.open3d(index, include_colors=True, max_num_points=None, include_normals=False) returns an Open3D point cloud. Colors are clamped to [0,1] and values above one are interpreted as 255-scale colors.
• Pointclouds.plotly(index, include_colors=True, max_num_points=200000, as_figure=True, point_size=2) returns a Plotly figure or Scatter3d.
• RGBDImages.plotly(index, include_depth=True, as_figure=True, ms_per_frame=50) creates RGB/depth animation frames. Convert to channels-last first and do not assume this is a GUI-free display operation.
• Adapter subsampling uses random permutations when a maximum is set; seed Torch if reproducibility matters.

Run the bundled helper for a tiny CPU fixture, then add focused assertions for any requested layout, pose, raggedness, missing-attribute, or adapter case. Do not replace this check with dataset downloads, notebook execution, GUI visualization, or a CUDA requirement.