Skip to content

Isaac Holoscan Bridge#

Authors: Holoscan Team (NVIDIA)
Supported platforms: x86_64
Language: Python
Last modified: June 25, 2025
Latest version: 0.1.0
Minimum Holoscan SDK version: 3.3.0
Tested Holoscan SDK versions: 3.3.0
Contribution metric: Level 3 - Developmental

The Isaac Holoscan Bridge is a sophisticated application that demonstrates the integration between NVIDIA's Isaac Simulator and the Holoscan framework.

isolated

It creates a bridge where simulated data, including camera and robotic joint data, from Isaac Sim is processed through a Holoscan pipeline. The application features a virtual camera in Isaac Sim that captures video frames, which are then streamed to a Holoscan pipeline for real-time processing. The processed data is then again displayed in Isaac Sim. The application also streams joint positions from a robot to a Holoscan pipeline. The pipeline modifies these joint positions and feeds them back into the simulation moving the robotic arm.

The pipeline includes several specialized operators: * an AsyncDataPushOp for handling data streaming from Isaac Sim to Holoscan * a CallbackOp which is handling data transfer from Holoscan to Isaac Sim * a SobelOp for image processing (edge detection) * a ControlOp for managing arm joint positions * and HolovizOp for visualization.

A unique feature of this application is its bidirectional callback system, where data flows between the simulation and Holoscan pipeline through carefully orchestrated callbacks, enabling real-time interaction between the two systems.

Flow Diagram#

flowchart LR
    subgraph Holoscan application
        AsyncDataPushOp -->|camera_image| HolovizOp
        AsyncDataPushOp -->|camera_image| SobelOp
        AsyncDataPushOp -->|arm_joint_positions| ControlOp
        SobelOp -->|camera_image_sobel| CallbackOp
        SobelOp -->|camera_image_sobel| HolovizOp
        ControlOp -->|arm_joint_positions| CallbackOp
    end
    subgraph Isaac Sim
        Simulation --> push_data_callback
        data_ready_callback --> Simulation
        CallbackOp -.->|callback| data_ready_callback
        push_data_callback -.->|callback| AsyncDataPushOp
    end

Requirements#

Run Instructions#

./dev_container build_and_run --container_launch_args "--as_root" --container_args "-e ACCEPT_EULA=Y -e PRIVACY_CONSENT=Y \
-v ${HOME}/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
-v ${HOME}/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
-v ${HOME}/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
-v ${HOME}/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
-v ${HOME}/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
-v ${HOME}/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
-v ${HOME}/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
-v ${HOME}/docker/isaac-sim/documents:/root/Documents:rw" isaac_holoscan_bridge

To keep Isaac Sim configuration and data persistent when running in a container, various directories are mounted into the container.

Note It takes quite a few minutes when this command is run the first time since shaders need to be compiled.