Skip to content

Reuse existing visual material in randomization#395

Open
yuecideng wants to merge 15 commits into
mainfrom
feat/reuse-existing-visual-material
Open

Reuse existing visual material in randomization#395
yuecideng wants to merge 15 commits into
mainfrom
feat/reuse-existing-visual-material

Conversation

@yuecideng

Copy link
Copy Markdown
Contributor

Description

This PR makes randomize_visual_material reuse each object's existing dexsim-parsed material (instance-swap + pre-created textures) instead of creating a new material each reset. It adds a three-tier texture choice (original / library / solid) with configurable probabilities, and preserves the old create-new-material behavior behind a fallback_to_new flag.

Motivation and context:

  • Preserve asset original appearance — randomization now perturbs on top of the material dexsim parsed from the mesh/URDF; the "original" tier restores it faithfully across resets, instead of replacing it with a blank white material.
  • Reduce per-reset overhead — library textures are pre-created once and cached at the sim level (no per-reset upload), and Env.clean_materials() is no longer called on the new path.

Key changes:

  • material.py: VisualMaterialInst.set_base_color_texture(texture_obj=...) to bind a pre-created Texture; ReuseSegmentState dataclass.
  • rigid_object.py / articulation.py: get_existing_visual_material / apply_render_material_inst.
  • visual.py: reuse path (_init_reuse / _call_reuse, three-tier swap, pre-created textures, tier-probability backward-compat derivation) + fallback_to_new legacy preservation + a get_data_path(None) guard so omitting texture_path no longer crashes.

Backward compatibility: fallback_to_new=True preserves the exact legacy path. When p_* are unset, they derive from the existing random_texture_prob (p_original=0), so existing configs run on the new path with zero changes. default_plane keeps its in-place behavior (no clean_materials in new mode).

Dependencies: none (uses existing dexsim APIs).

Design spec: docs/superpowers/specs/2026-07-14-reuse-existing-visual-material-design.md
Implementation plan: docs/superpowers/plans/2026-07-14-reuse-existing-visual-material.md

Known limitations / follow-ups (Minor, logged in .superpowers/sdd/progress.md): type hints on private helpers; _call_legacy docstring; test-quality polish. Non-shared _call_reuse assumes full-reset len(env_ids)==num_reuse (matches the legacy path's assumption).

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which improves an existing functionality)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Screenshots

N/A — simulation material behavior, verified via 24 unit tests with mocked dexsim (no GPU required).

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • Dependencies have been updated, if applicable.

🤖 Generated with Claude Code

yuecideng and others added 15 commits July 13, 2026 19:12
Design for randomize_visual_material to reuse the object's existing
dexsim-parsed material (instance-swap + pre-created textures) instead of
creating a new material each time. Preserves asset original appearance
(three-tier: original/library/solid) and removes per-reset material
creation and clean_materials overhead. Old behavior retained behind a
fallback_to_new flag.

Co-Authored-By: Claude <noreply@anthropic.com>
7-task TDD plan for instance-swap + pre-created-texture reuse path in
randomize_visual_material, with fallback_to_new legacy preservation.

Co-Authored-By: Claude <noreply@anthropic.com>
Pre-flight fix: solid/library tier logic was duplicated between rigid and
articulation paths. Helpers now take link_name and share _apply_inst.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…_material_inst

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…_material_inst

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
… swap

Co-Authored-By: Claude <noreply@anthropic.com>
…ure_key

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 14, 2026 01:59
@yuecideng yuecideng added enhancement New feature or request dexsim Things related to dexsim rendering Things related to rendering (eg, performace, efficiency, bug) object Simulation object assets labels Jul 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors randomize_visual_material to reuse dexsim-parsed per-object materials (via instance swapping) instead of creating and replacing a new material each reset, adds a three-tier texture selection policy (original/library/solid) with configurable probabilities, and keeps the prior behavior behind a fallback_to_new flag. It also introduces supporting sim APIs (ReuseSegmentState, “get existing material” helpers) and adds mocked unit tests to validate the new code paths.

Changes:

  • Extend VisualMaterialInst.set_base_color_texture(...) to accept a pre-created dexsim Texture object (skip re-upload).
  • Add ReuseSegmentState plus get_existing_visual_material(...) / apply_render_material_inst(...) helpers on RigidObject and Articulation.
  • Implement reuse-mode init/call paths in randomize_visual_material, including tier-probability resolution and sim-level texture object caching; add extensive mocked tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
embodichain/lab/gym/envs/managers/randomization/visual.py Implements reuse-mode initialization and per-reset swapping, tier sampling, and cached texture objects; retains legacy path behind flag.
embodichain/lab/sim/material.py Adds texture_obj support for base-color maps and introduces ReuseSegmentState.
embodichain/lab/sim/objects/rigid_object.py Adds helpers to capture existing segment materials and swap MaterialInst back onto render bodies.
embodichain/lab/sim/objects/articulation.py Adds link-aware equivalents of existing-material capture and render-body instance swapping.
embodichain/lab/sim/__init__.py Exports ReuseSegmentState and defines an explicit package __all__.
tests/gym/envs/managers/test_randomize_visual_material.py Adds characterization and reuse-mode tests (mocked dexsim) for legacy/reuse/compat scenarios.
tests/sim/test_material_texture_obj.py Tests texture_obj path binds Texture without upload.
tests/sim/objects/test_rigid_object_reuse_material.py Tests rigid-object reuse-state building and render-body swapping.
tests/sim/objects/test_articulation_reuse_material.py Tests articulation reuse-state building per link and swapping per link segment.
docs/superpowers/specs/2026-07-14-reuse-existing-visual-material-design.md Design write-up for the reuse-material approach and constraints.
docs/superpowers/plans/2026-07-14-reuse-existing-visual-material.md Detailed implementation plan and checklist for the feature.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +640 to +644
logger.log_warning(
f"randomize_visual_material: reuse-existing-material unavailable for "
f"'{self.entity_cfg.uid}' ({e}); falling back to new-material path."
)
self._new_mode = False
Comment on lines +872 to +875
else:
for reuse_i in range(num_reuse):
_apply(reuse_i, int(env_ids[reuse_i]))

Comment on lines +191 to +193
seg = _seg(0, MagicMock(name="orig"), MagicMock(name="tmpl"))
obj.get_existing_visual_material = MagicMock(return_value=[[seg]])
cfg = _make_cfg({"entity_cfg": SceneEntityCfg(uid="obj")})
functor(env, torch.arange(env.num_envs), entity_cfg=SceneEntityCfg(uid="obj"))

env.sim.env.clean_materials.assert_not_called()
obj.apply_render_material_inst.assert_called()
Comment on lines +253 to 256
texture_path: str = None,
texture_data: torch.Tensor | None = None,
texture_obj=None,
) -> None:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dexsim Things related to dexsim enhancement New feature or request object Simulation object assets rendering Things related to rendering (eg, performace, efficiency, bug)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants