---

name: "simulation-rendering" description: "Use for MuJoCo model loading, simulation state and time inspection, headless or onscreen rendering, camera/output configuration, and viewer/display diagnostics; route environment catalog, XML editing/IK, MJX acceleration, and training elsewhere." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Simulation and rendering

Use this sub-skill when the task concerns the MuJoCo model/data handles behind a MyoSuite environment, rendering pixels or depth, selecting cameras, preserving or restoring simulator state, or diagnosing a viewer/display failure. It covers the base MuJoCo backend in MyoSuite 2.x. It does not teach task selection, XML mutation or IK, MJX/JAX acceleration, or RL training.

Route first

• Choose environment workflows for registry lookup, task configuration, reset/step semantics, or environment-specific observation and reward behavior.
• Choose model editing and kinematics for XML changes, MjSpec editing, site/body edits as a modeling operation, or IK.
• Choose MJX acceleration for JAX, MJX, CUDA, batching, or accelerator performance.
• Choose training integration for policies, learners, checkpoints, or long-running experiments.

Fast decision: which rendering path?

1. Headless or CI: load mujoco.MjModel/mujoco.MjData, step with mujoco.mj_step, and use MJRenderer.render_offscreen(...), or run the bundled safe checker: python scripts/check_mujoco_xml.py --help.
2. A window is explicitly wanted: env.mj_render() delegates to env.mj_renderer.render_to_window(), which creates a native passive viewer on first use. This is display-dependent and is not verification ground truth.
3. Raw XML diagnosis: run python scripts/check_mujoco_xml.py --xml MODEL.xml --render none first. It loads and steps without creating a viewer. Add --render offscreen only when pixel output is required.

Never use myosuite.utils.examine_sim as a headless check: its documented workflow calls mujoco.viewer.launch(...) and is intentionally onscreen. Its safe native candidate is python -m myosuite.utils.examine_sim --help only.

Environment rendering contract

from myosuite.utils import gym

env = gym.make("myoElbowPose1D6MRandom-v0")
env.reset(seed=1234)
frame = env.mj_renderer.render_offscreen(
    width=320, height=240, camera_id=-1, rgb=True
)
env.close()

• env.mj_model is the compiled mujoco.MjModel; env.mj_data is its live mujoco.MjData. The base environment constructs env.mj_renderer as an MJRenderer for those handles.
• camera_id=-1 means MuJoCo's free camera. A named camera string or numeric camera id selects a model camera. Use a model camera name only after checking that it exists; a missing name is a model/rendering error, not a display fix.
• render_offscreen returns an RGB numpy array for the default call. The detailed return matrix for RGB/depth/segmentation is in rendering-api.md.
• env.mj_render() is deliberately different: it opens/synchronizes a window and returns no frame. Do not call it in a server, CI job, or headless synthetic case.
• env.viewer_setup(distance=..., azimuth=..., elevation=..., lookat=..., render_actuator=..., render_tendon=...) stores free-camera and visualization settings for the next window or offscreen scene. Camera/output details and the distance adjustment are documented in rendering-api.md.

Deterministic headless rollout

For a raw model, the safe contract is:

python scripts/check_mujoco_xml.py \
  --xml MODEL.xml --render offscreen --frames 8 \
  --width 320 --height 240 --output-dir ./mujoco-check

The command never calls mujoco.viewer or launch_passive. It prints model sizes, steps the requested number of frames, and writes one binary PPM image per rendered frame under the supplied output directory (for example, mujoco-check/frame-0000.ppm). --render none is the default and creates no output directory. The script accepts optional comma-separated --qpos and --ctrl; it validates their lengths before applying them. See the script's --help output for the complete input/output contract.

For a policy/environment rollout, use MujocoEnv.examine_policy_new only when its policy API is already available. Its supported render values are "onscreen", "offscreen", and "none"; offscreen mode accumulates RGB frames and writes an MP4 per episode through imageio. Set an explicit output_dir, filename, frame_size, and finite horizon. The safe raw-model script is preferred for a small, dependency-light rendering check.

State and time workflow

• env.dt is env.mj_model.opt.timestep * env.frame_skip.
• env.time reads the observed simulation data time (env.obsd_mj_data.time), so compare it with env.mj_data.time only when the environment uses the same observed and ground-truth model.
• env.get_env_state() returns copied time, qpos, qvel, optional act, optional mocap position/quaternion, optional site position/quaternion, and body position/quaternion arrays.
• Save the returned dictionary before a branch, then call env.set_env_state(state) to restore the base state. The method updates both ground-truth and observed data where present and performs a MuJoCo step to refresh derived state; treat restoration as a simulator-state operation, not as a byte-for-byte snapshot of every internal buffer.
• env.reset(...) establishes the task's initial state. After changing raw qpos/qvel, use mujoco.mj_forward(model, data) before inspecting derived positions or rendering. Do not edit model arrays merely to move a body unless the task is explicitly a model-editing task.

Visual observations

Visual observations are opt-in. Call env.get_obs(update_exteroception=True) or env.get_visuals(...) when the environment has configured visual_keys. Supported key forms are rgb:CAMERA:HxW:1d, rgb:CAMERA:HxW:2d, and optional encoder forms such as r3m18, r3m34, r3m50, rrl, or vc1 when their optional encoder dependencies are installed. A matching d: key requests depth alongside the RGB key. get_obs() does not refresh exteroception by default, and env_info["visual_dict"] may therefore be empty or stale unless visuals were explicitly updated at the current simulation time.

Backend and safety boundary

Base MuJoCo model loading, stepping, state inspection, and the MJRenderer offscreen path are CPU/base-package capabilities, subject to a usable graphics backend for pixel rendering. Onscreen viewing additionally requires a display and native viewer support. MJX/JAX/CUDA is optional and is not established by an offscreen CPU check; route it to the MJX sub-skill and report it separately.

Before reporting success, check the actual artifact: model load/step succeeded, rendered arrays have the requested dimensions, and every requested output file exists and is non-empty. For failure symptoms and recovery branches, use troubleshooting.md.