---

name: "robomind-conversion" description: "Guides RoboMIND benchmark and embodiment HDF5 trees into LeRobot datasets, including language annotations, image/depth decoding, configuration mapping, safe execution choices, and recovery of conversion failures." disable-model-invocation: true metadata: disco-role: operating license: MIT

RoboMIND conversion

Use this route when a user needs to convert RoboMIND benchmark data stored as per-embodiment HDF5 episode trees into local LeRobot datasets. It covers the RoboMIND-specific directory contract, task language, state/action features, RGB/depth decoding, embodiment selection, and safe execution planning.

This route does not run a conversion, create a Ray cluster, download data, push to a Hub, or render simulation data while drafting or validating a plan. It also does not cover AgiBot, LIBERO, RoboCasa, generic adapter execution, or LeRobot version migration; route those requests to their owning sub-skills.

Read before planning

• Workflow and execution choices
• Input and output data formats
• Embodiment and feature catalog
• Troubleshooting and recovery

Safety and applicability

• Accept a local, readable RoboMIND release root and a distinct local output root. Treat both paths as user data, not as disposable test fixtures.
• Require one of the three supported benchmark names and one or more of the eight physical embodiment names. The simulation labels in the configuration registry are intentionally not a supported CLI route.
• Validate the complete hierarchy, HDF5 keys, frame counts, image byte sizes, and language annotation coverage before scheduling any episode work.
• Prefer a single-task --debug smoke plan. The normal path initializes Ray and can schedule many tasks concurrently; never select it accidentally merely because Ray is installed.
• The converter removes an existing per-task output directory before writing. Require an explicit output inventory and approval before replacing anything.
• Keep depth optional and verify the installed LeRobot image-stat behavior for one-channel depth before a real run. Do not silently drop depth or reshape incompatible images to satisfy a declared feature shape.

Route checklist

1. Freeze source root, benchmark, embodiment list, output root, depth choice, and desired split policy. Record the RoboMIND/LeRobot compatibility versions.
2. Check the release tree and the CSV/JSON annotation files described in data formats. Reject ambiguous task names instead of guessing an instruction.
3. Select the exact config for every embodiment and validate that all configured puppet/*, master/*, RGB, and optional depth datasets exist with aligned lengths. Use embodiments for widths and shapes.
4. Decide debug versus Ray. Debug processes only the first discovered task for the first selected embodiment and avoids Ray; it is the baseline smoke path. Ray is an approved throughput option only after resource and cluster ownership are explicit.
5. Plan output and error logging. The implementation writes each task under output/benchmark/embodiment/task and appends Ray failures to output.txt in the process working directory; make that location intentional.
6. Convert only after a preflight report records skipped episodes, dirty-task decisions, annotation fallbacks, image decoding decisions, and output replacement approval.
7. Inspect meta/info.json, episode metadata, train/validation ranges, feature shapes, action metadata, frame counts, and logs before any downstream use or publication.

RoboMIND-specific behavior

The resulting dataset uses v3-style metadata with fps: 30, a robot_type matching the embodiment, dictionary-like observation.states.* and actions.* features, and video/image features generated from the selected config. Natural-language task text is attached to every frame. Episode-level JSON responses are stored as action_config metadata when a matching annotation exists; missing matches use a null-valued fallback and must be reported.

RGB streams for franka_1rgb, franka_3rgb, franka_fr3_dual, and ur_1rgb are decoded as BGR by the evidence workflow and must be converted to RGB. Other embodiments are not given that conversion. A failed decode may use only the known raw byte-size shape fallbacks; any other shape is a data error. The published implementation also has a mutable top-camera 720x1280 ↔ 480x640 retry; treat that as a bounded, explicitly recorded fallback, not a general repair mechanism. See troubleshooting.

Verification boundary and handoff

Safe verification is limited to static link/config checks, CLI parser/help inspection when imports are available, and a tiny synthetic directory/config model that does not open real HDF5, encode video, write LeRobot data, start Ray, or allocate large arrays. A difficult case must cover franka_3rgb BGR input, malformed image shape, and the explicit shape fallback without launching Ray.

Handoff should include the selected benchmark and embodiments, source/output roots, split and depth decisions, config and annotation validation, debug or Ray mode, CPU/memory budget, skipped/dirty episodes, output replacement status, metadata checks, log locations, and unresolved compatibility issues.

Evidence basis: the RoboMIND README, robomind_h5.py, robomind_uitls.py, lerobot_uitls.py, and the eight embodiment configuration modules. These artifacts are provenance only; this skill contains the distilled contract and has no runtime dependency on the source checkout.