Skip to content

Laser Detection Latency

Authors: Holoscan Team (NVIDIA)
Supported platforms: aarch64
Last modified: March 18, 2025
Language: Python
Latest version: 1.0
Minimum Holoscan SDK version: 2.1.0
Tested Holoscan SDK versions: 2.2.0
Contribution metric: Level 1 - Highly Reliable

Introduction

The laser detection latency application demonstrates the latency differences between two different cameras visually.

This folder contains three applications, please refer to the respective application folder for application and configuration file.

  1. USB Camera Calibration app: This app is designed to do monitor registration by the USB camera. It uses a gray image which has April tags on all the four corners and is as shown below.

The detection of these four April tags are done using ApriltagDetectionOp. For proper and correct monitor registration, the camera should be able to completely see all the four corners of the monitor. If it does not, the app will not output correct corners for the monitor. It is also important to make sure that the scene is well lit with no light sources in the back of the monitor.

This app is designed using Logitech 4k Pro Webcam. If a different camera is being used, please change the camera settings in the python app or yaml configuration file.

  1. EVT Camera Calibration app: This app is designed to do monitor registration by the Emergent Vision Technologies (EVT) camera. It uses the same gray image which has April tags on all the four corners as shown above.

In this app the detection of these four April tags are done using ApriltagDetectionOp as well. For proper and correct monitor registration, the camera should be able to completely see all the four corners of the monitor. If it does not, the app will not output correct corners for the monitor. It is also important to make sure that the scene is well lit with no light sources in the back of the monitor.

This app is designed using EVT HB-9000-G 25GE. If a different camera is being used, please change the camera settings in the python app or yaml configuration file.

  1. Laser Detection app: The laser detection is the app that is run after the calibration apps detected the monitor successfully. This app uses two camera sources: one is EVT camera and other is USB camera. The video feed from both camera is used to detect laser pointed at the monitor. There are two icons that will be shown on the display. The white icon represents USB camera and the green icon represents the EVT camera. When the laser is detected the respective icons move to the coordinates. The laser detection algorithm is same for both camera sources.

Hardware requirements

1. USB camera

The app is designed using Logitech 4k Pro Webcam. A different webcam can also be used, but if resolution settings are different the application code and yaml file will need to be updated accordingly.

2. EVT camera

Visit Holoscan SDK user guide to check the hardware requirements for EVT camera.

3. Monitor

Any matte screen monitor with a refresh rate of at least 120fps is required to see the performance differences between the two camera sources.

4. Laser pointer

Please make sure to only buy laser pointers that are safe to use for viewing purposes.

Setting up IGX Development Kit

1. Setup the EVT camera

To setup the EVT camera, refer to Holoscan SDK user guide.

2. Build Holohub app natively

Currently EVT camera based apps are only available for native environment. To build the setup natively, download the Holohub repo and run following.

sudo ./run setup

3. Install Python3 requirements

sudo pip3 install -r applications/laser_detection_app/requirements.txt

4. Install CVCUDA

Download and install the latest CVCUDA package. Replace the latest version in the below commands. 

wget -O cvcuda-lib-0.10.1_beta-cuda12-aarch64-linux.deb https://github.com/CVCUDA/CV-CUDA/releases/download/v0.10.1-beta/cvcuda-lib-0.10.1_beta-cuda12-aarch64-linux.deb
wget -O cvcuda-python3.10-0.10.1_beta-cuda12-aarch64-linux.deb https://github.com/CVCUDA/CV-CUDA/releases/download/v0.10.1-beta/cvcuda-python3.10-0.10.1_beta-cuda12-aarch64-linux.deb
sudo dpkg -i cvcuda-lib-0.10.1_beta-cuda12-aarch64-linux.deb
sudo dpkg -i cvcuda-python3.10-0.10.1_beta-cuda12-aarch64-linux.deb

5. Setup cuApriltags

Download the latest cuApriltags library and header file, and set it up on IGX Orin as follows: 

sudo mkdir /opt/nvidia/cu-april-tags
sudo mkdir /opt/nvidia/cu-april-tags/include
sudo mkdir /opt/nvidia/cu-april-tags/lib
sudo cp libcuapriltags.a /opt/nvidia/cu-april-tags/lib/.
sudo cp cuAprilTags.h /opt/nvidia/cu-april-tags/include/.

Running the sample apps

1. Build and run usb_cam_calibration

Before running the app, make sure that the USB camera can see all the corners of the monitor. The v4l2_camera app can be used to verify it visually.

./run build usb_cam_calibration
LD_PRELOAD=/usr/lib/aarch64-linux-gnu/nvidia/libnvjpeg.so ./run launch usb_cam_calibration
This will output a file usb-cali.npy in the build directory.

2. Build and run evt_cam_calibration

Before running the app, make sure that the EVT camera can see all the corners of the monitor. The high_speed_endoscopy app can be used to verify it visually. 

./run build evt_cam_calibration
sudo ./run launch evt_cam_calibration
This will output a file evt-cali.npy in the build directory.

note:

Use sudo when running the application with EVT camera.

3. Build and run laser_detection

./run build laser_detection
sudo LD_PRELOAD=/usr/lib/aarch64-linux-gnu/nvidia/libnvjpeg.so ./run launch laser_detection
Now you can use the laser pointer and point it to the monitor. If the icons are not detecting the laser or they are moving in random fashion, then redo the calibration steps.

note:

  • Please make sure that the monitor that is being used has matte screen. The glossy screen can create specular reflections and could damage the eye of the person looking at the monitor at certain angles.
  • Please make sure to only buy laser pointers that are safe to use for viewing purposes.