Skip to content

XR + Gaussian Splatting#

Authors: Connor Smith (NVIDIA), Mimi Liao (NVIDIA)
Supported platforms: x86_64
Language: Python
Last modified: March 9, 2026
Latest version: 1.0.0
Minimum Holoscan SDK version: 3.1.0
Tested Holoscan SDK versions: 3.1.0
Contribution metric: Level 4 - Experimental

This application demonstrates rendering a 3D scene using Gaussian Splatting in XR.

Demo Demo running on Apple Vision Pro

Flow Diagram#

fig Diagram illustrating the architecture and data flow of this application

0. Training a Gaussian Splatting Model#

The below instructions are based on the gsplat colmap example.

0.1. Clone the gsplat repo#

git clone https://github.com/nerfstudio-project/gsplat.git

0.2. Install dependencies and download the data#

cd gsplat/examples
# Install torch
pip install "torch>=2.10.0"
# Install gsplat
pip install git+https://github.com/nerfstudio-project/gsplat.git
# Install dependencies
pip install -r requirements.txt
# Download the data
python datasets/download_dataset.py

0.3. Train the model#

CUDA_VISIBLE_DEVICES=0 python simple_trainer.py default \
    --data_dir data/360_v2/garden/ --data_factor 4 \
    --result_dir ./results/garden

note: Training time is observed to take about 30 minutes on Intel i9 CPU + NVIDIA RTX A5000 dGPU

0.4. Set up the checkpoint paths in config.yaml#

Quick Start#

Option 1: Stream to Apple Vision Pro (CloudXR)#

To stream this XR application to devices like Apple Vision Pro, refer to the CloudXR Runtime tutorial for setup instructions.

Option 2: Using Monado OpenXR Runtime (Local Development)#

Monado is an open-source OpenXR runtime that supports various XR devices and includes a simulator for local development without physical XR hardware.

Terminal 1: Launch Container and Start Monado Service#

# If you're already in the container, skip this step
./holohub run-container xr_gsplat

# Inside the container, start the Monado XR runtime service
monado-service

Keep this terminal open and running.

Terminal 2: Build and Run the Application#

# Enter the same container (replace <container_id> with actual ID from 'docker ps')
docker exec -it <container_id> bash

# Build and run the application
./holohub run xr_gsplat

Checkpoint tests#

Run the application's checkpoint regression suite through CTest:

./holohub test xr_gsplat --language python

The suite loads real gsplat training state dictionaries and rejects executable pickle payloads in both ZIP and legacy checkpoint formats. It uses generated inputs and does not require downloaded models or an XR runtime. These tests exercise checkpoint loading, not full rendering.

To run pytest directly from the repository root in an environment with the application dependencies and pytest installed:

python -m pytest applications/xr_gsplat/test_gsplat_loader_op.py