Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion agent_context/topics/randomization/randomization.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,16 @@ The `__init__.py` of the randomization package re-exports everything via `from .
| `randomize_visual_material` | Random material properties | (varies) |
| `randomize_camera_extrinsics` | Camera pose | `pos_range`, `euler_range` (attach mode) or `eye_range`, `target_range`, `up_range` (look-at mode) |
| `randomize_camera_intrinsics` | Camera intrinsic params | (varies) |
| `randomize_light` | Light pos/color/intensity | `position_range`, `color_range`, `intensity_range` |
| `randomize_light` | Light pos/color/intensity/direction | `position_range`, `color_range`, `intensity_range`, `direction_range` |
| `randomize_emission_light` | Emission light props | (varies) |
| `randomize_indirect_lighting` | Indirect lighting | (varies) |

- Camera extrinsics auto-detect mode: if `extrinsics.parent` is set → attach mode (pos/euler perturbation via `set_local_pose`); if `extrinsics.eye` is set → look-at mode (eye/target/up perturbation via `look_at`).
- `set_rigid_object_visual_material` is deterministic (not random) but uses the same functor mechanism for fixed material assignment at reset.
- Light randomization applies the **same values across all envs** (documented limitation).
- ``position_range`` is ignored for global scene lights (``"sun"``, ``"direction"``). Use ``direction_range`` instead.
- ``direction_range`` is only applicable for directional light types (``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, ``"mesh"``).
- Global lights (``"sun"``, ``"direction"``) have ``num_instances == 1``; all other types are batched per environment.

### Spatial (`spatial.py`)

Expand Down
13 changes: 11 additions & 2 deletions docs/source/overview/gym/event_functors.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,24 @@ This page lists all available event functors that can be used with the Event Man
"base_color_range": [[0.2, 0.2, 0.2], [1.0, 1.0, 1.0]]}}
```
* - {func}`~randomization.visual.randomize_light`
- Vary light position, color, and intensity within specified ranges.
- Vary light position, color, intensity, and direction within specified ranges.

.. note::
``position_range`` is ignored for global scene lights (``"sun"``, ``"direction"``).
Use ``direction_range`` instead for these types.

.. note::
``direction_range`` is only applicable for directional light types
(``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, ``"mesh"``).

```json
{"func": "randomize_light",
"mode": "interval", "interval_step": 10,
"params": {"entity_cfg": {"uid": "light_1"},
"position_range": [[-0.5, -0.5, 2], [0.5, 0.5, 2]],
"color_range": [[0.6, 0.6, 0.6], [1, 1, 1]],
"intensity_range": [50.0, 100.0]}}
"intensity_range": [50.0, 100.0],
"direction_range": [[-0.1, -0.1, -0.1], [0.1, 0.1, 0.1]]}}
```
* - {func}`~randomization.visual.randomize_emission_light`
- Randomize global emission light color and intensity. Applies the same emission light properties across all environments.
Expand Down
23 changes: 18 additions & 5 deletions docs/source/overview/sim/sim_assets.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,14 +112,27 @@ For Rigid Object tutorial, please refer to the [Create Scene](https://dexforce.g

### Lights

Configured via `LightCfg`.
Configured via `LightCfg`. Supports six light types matching the dexsim rendering backend.

| Parameter | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `light_type` | `Literal` | `"point"` | Type of light (currently only "point"). |
| `color` | `tuple` | `(1.0, 1.0, 1.0)` | RGB color. |
| `intensity` | `float` | `50.0` | Intensity in watts/m^2. |
| `radius` | `float` | `1e2` | Falloff radius. |
| `light_type` | `Literal` | `"point"` | Light type: ``"point"``, ``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, or ``"mesh"``. |
| `color` | `tuple` | `(1.0, 1.0, 1.0)` | RGB color of the light source. |
| `intensity` | `float` | `30.0` | Intensity of the light source in watts/m^2. |
| `enable_shadow` | `bool` | `True` | Whether the light casts shadows. |
| `radius` | `float` | `10.0` | Falloff radius (only for ``"point"``). |
| `direction` | `tuple` | `(0.0, 0.0, -1.0)` | Direction vector for directional types (``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, ``"mesh"``). |
| `spot_angle_inner` | `float` | `30.0` | Inner cone angle in degrees (only for ``"spot"``). |
| `spot_angle_outer` | `float` | `45.0` | Outer cone angle in degrees (only for ``"spot"``). |
| `rect_width` | `float` | `1.0` | Width of rectangular area light (only for ``"rect"``). |
| `rect_height` | `float` | `1.0` | Height of rectangular area light (only for ``"rect"``). |
| `mesh_path` | `str` | `""` | Asset path for mesh-based emissive lights (only for ``"mesh"``). |

.. attention::
``"sun"`` and ``"direction"`` are **global scene lights** (infinite-distance directional light
sources). They are created as a single instance on the root environment, not batched per
environment. Use :meth:`Light.set_direction` instead of :meth:`Light.set_local_pose` for
these types.


```{toctree}
Expand Down
73 changes: 73 additions & 0 deletions docs/source/overview/sim/sim_manager.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,79 @@ sim_config = SimulationManagerCfg(
)
```

### DLSS 3.5 (OfflineRT Only)

When `render_cfg.renderer="rt"` (OfflineRT) and the simulation is running with a visible window, you can enable NVIDIA DLSS 3.5 for AI-powered denoising and upscaling through `render_cfg.dlss`.

DLSS 3.5 combines two features:

- **Ray Reconstruction (RR)** — replaces the OptiX denoiser for the window camera.
- **Super Resolution (SR)** — upscales the RR output from render resolution to the target/display resolution.

Both features are enabled together when `dlss_enabled=True`. The exact upscale ratio can be controlled either with the `dlss_quality` preset or with the `upsample_ratio` convenience parameter.

| Parameter | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `dlss_enabled` | `bool` | `True` | Master DLSS enable toggle. |
| `dlss_quality` | `int` | `-1` | DLSS quality preset. `-1`=auto, `0`=UltraPerformance (~3.0×), `1`=Performance (~2.25×), `2`=Balanced (~1.75×), `3`=Quality (~1.5×), `4`=UltraQuality (~1.3×), `5`=DLAA (1.0×). |
| `upsample_ratio` | `float` | `1.0` | Convenience ratio `target_width / render_width` (and height). When `render_width`/`render_height` are `0`, the render resolution is computed as `target / upsample_ratio`. Defaults to `1.0` (1:1 / DLAA mode). Must be `>= 1.0`. |
| `render_width` | `int` | `0` | Internal render resolution width. `0` means use the effective target width. |
| `render_height` | `int` | `0` | Internal render resolution height. `0` means use the effective target height. |
| `target_width` | `int` | `0` | Target/display resolution width. `0` means use the window width. |
| `target_height` | `int` | `0` | Target/display resolution height. `0` means use the window height. |
| `exposure_compensation` | `float` | `1.0` | Multiplier on the DLSS-RR pre-exposure scalar. `>1` brightens the HDR input to RR, which helps reject ghosting in dark areas; `<1` darkens. Typical range `0.5–8.0`. |

#### Resolution selection rules

- The **target** resolution defaults to the window size unless `target_width`/`target_height` are set explicitly.
- The **render** resolution defaults to the target resolution (1:1 / DLAA mode) unless overridden.
- If `upsample_ratio > 1.0` and `render_width`/`render_height` are `0`, the render resolution is computed as `target / upsample_ratio`. At `1.0`, render defaults to the target resolution (1:1 / DLAA mode).
- Explicit `render_width`/`render_height` or `target_width`/`target_height` take precedence over computed values.
- The window is always resized to match the effective target resolution when DLSS is active.

#### Examples

Use the `upsample_ratio` shortcut to render at half resolution and upscale to the window size:

```python
from embodichain.lab.sim import SimulationManagerCfg
from embodichain.lab.sim.cfg import DLSSCfg, RenderCfg

sim_config = SimulationManagerCfg(
width=1920,
height=1080,
render_cfg=RenderCfg(
renderer="rt",
dlss=DLSSCfg(
dlss_enabled=True,
upsample_ratio=2.0, # render at 960x540, upscale to 1920x1080
),
),
)
```

Set explicit render and target resolutions:

```python
from embodichain.lab.sim import SimulationManagerCfg
from embodichain.lab.sim.cfg import DLSSCfg, RenderCfg

sim_config = SimulationManagerCfg(
render_cfg=RenderCfg(
renderer="rt",
dlss=DLSSCfg(
dlss_enabled=True,
render_width=1280,
render_height=720,
target_width=1920,
target_height=1080,
),
),
)
```

> **Note:** Offscreen cameras always use the OptiX denoiser regardless of DLSS settings. DLSS only affects the interactive viewer window when the OfflineRT renderer is used.


## Initialization

Expand Down
9 changes: 5 additions & 4 deletions docs/source/tutorial/modular_env.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,11 +129,12 @@ Configures a stereo camera system using :class:`StereoCameraCfg`:
:language: python
:lines: 120-130

Defines scene illumination with controllable point lights:
Defines scene illumination with configurable lights:

- **Type**: Point light for realistic shadows
- **Properties**: Configurable color, intensity, and position
- **UID**: Named reference for event system manipulation
- **Types**: Supports ``"point"``, ``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, and ``"mesh"``.
- **Global lights**: ``"sun"`` and ``"direction"`` are global scene lights (single instance, infinite distance).
- **Properties**: Configurable color, intensity, position, and direction.
- **UID**: Named reference for event system manipulation.

**Rigid Objects**

Expand Down
84 changes: 68 additions & 16 deletions embodichain/lab/gym/envs/managers/randomization/visual.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,7 @@ def randomize_light(
position_range: tuple[list[float], list[float]] | None = None,
color_range: tuple[list[float], list[float]] | None = None,
intensity_range: tuple[float, float] | None = None,
direction_range: tuple[list[float], list[float]] | None = None,
) -> None:
"""Randomize light properties by adding, scaling, or setting random values.

Expand All @@ -289,36 +290,63 @@ def randomize_light(
position_range is the x, y, z value added into light's cfg.init_pos.
color_range is the absolute r, g, b value set to the light object.
intensity_range is the value added into light's cfg.intensity.
direction_range is the x, y, z value added into light's cfg.direction.
(Only applicable for ``"sun"``, ``"direction"``, ``"spot"``, ``"rect"``, and ``"mesh"`` light types.)

.. tip::
This function uses CPU tensors to assign light properties.

.. warning::
``position_range`` is ignored for global scene lights (``"sun"``, ``"direction"``)
because they are infinite-distance lights with no meaningful position.
Use ``direction_range`` instead for these light types.

Args:
env (EmbodiedEnv): The environment instance.
env_ids (Union[torch.Tensor, None]): The environment IDs to apply the randomization.
entity_cfg (SceneEntityCfg): The configuration of the scene entity to randomize.
position_range (tuple[list[float], list[float]] | None): The range for the position randomization.
color_range (tuple[list[float], list[float]] | None): The range for the color randomization.
intensity_range (tuple[float, float] | None): The range for the intensity randomization.
direction_range (tuple[list[float], list[float]] | None): The range for the direction randomization.
Only applicable for directional light types (``"sun"``, ``"direction"``, ``"spot"``,
``"rect"``, ``"mesh"``).
"""

light: Light = env.sim.get_light(entity_cfg.uid)
num_instance = len(env_ids)
if light is None:
return

is_global = light.is_global if hasattr(light, "is_global") else False

# For global lights, normalize env_ids to avoid index-out-of-range
if is_global or light.num_instances == 1:
num_instance = 1
effective_env_ids = None
else:
num_instance = len(env_ids)
effective_env_ids = env_ids

if position_range:
init_pos = light.cfg.init_pos
new_pos = (
torch.tensor(init_pos, dtype=torch.float32)
.unsqueeze_(0)
.repeat(num_instance, 1)
)
random_value = sample_uniform(
lower=torch.tensor(position_range[0]),
upper=torch.tensor(position_range[1]),
size=new_pos.shape,
)
new_pos += random_value
light.set_local_pose(new_pos, env_ids=env_ids)
if is_global:
logger.log_warning(
f"position_range ignored for global light '{entity_cfg.uid}' "
f"(type='{light.cfg.light_type}'). Use direction_range instead."
)
else:
init_pos = light.cfg.init_pos
new_pos = (
torch.tensor(init_pos, dtype=torch.float32)
.unsqueeze_(0)
.repeat(num_instance, 1)
)
random_value = sample_uniform(
lower=torch.tensor(position_range[0]),
upper=torch.tensor(position_range[1]),
size=new_pos.shape,
)
new_pos += random_value
light.set_local_pose(new_pos, env_ids=effective_env_ids)

if color_range:
color = torch.zeros((num_instance, 3), dtype=torch.float32)
Expand All @@ -328,7 +356,7 @@ def randomize_light(
size=color.shape,
)
color += random_value
light.set_color(color, env_ids=env_ids)
light.set_color(color, env_ids=effective_env_ids)

if intensity_range:
init_intensity = light.cfg.intensity
Expand All @@ -344,7 +372,31 @@ def randomize_light(
)
new_intensity += random_value
new_intensity.squeeze_(1)
light.set_intensity(new_intensity, env_ids=env_ids)
light.set_intensity(new_intensity, env_ids=effective_env_ids)

if direction_range:
light_type = light.cfg.light_type
if light_type not in ("sun", "direction", "spot", "rect", "mesh"):
logger.log_warning(
f"direction_range ignored for light '{entity_cfg.uid}' "
f"(type='{light_type}'). Direction only applicable to "
f"'sun', 'direction', 'spot', 'rect', and 'mesh' types."
)
return

init_dir = light.cfg.direction
new_dir = (
torch.tensor(init_dir, dtype=torch.float32)
.unsqueeze_(0)
.repeat(num_instance, 1)
)
random_value = sample_uniform(
lower=torch.tensor(direction_range[0]),
upper=torch.tensor(direction_range[1]),
size=new_dir.shape,
)
new_dir += random_value
light.set_direction(new_dir, env_ids=effective_env_ids)


def randomize_emission_light(
Expand Down
Loading