H.264 Endoscopy Tool Tracking Application ¶

Authors: Holoscan Team (NVIDIA)
Supported platforms: x86_64, aarch64
Last modified: March 18, 2025
Language: Python
Latest version: 1.0
Minimum Holoscan SDK version: 2.6.0
Tested Holoscan SDK versions: 2.6.0
Contribution metric: Level 1 - Highly Reliable

The application showcases how to use H.264 video source as input to and output from the Holoscan pipeline. This application is a modified version of Endoscopy Tool Tracking reference application in Holoscan SDK that supports H.264 elementary streams as the input and output.

The H.264 video decode operators do not adjust framerate as it reads the elementary stream input. As a result the video stream can be displayed as quickly as the decoding can be performed. This application uses PeriodicCondition to play video at the same speed as the source video.

Requirements¶

This application is configured to use H.264 elementary stream from endoscopy sample data as input. The recording of the output can be enabled by setting record_output flag in the config file to true. If the record_output flag in the config file is set to true, the output of the pipeline is again recorded to a H.264 elementary stream on the disk, file name / path for this can be specified in the 'h264_endoscopy_tool_tracking.yaml' file.

Data¶

📦️ (NGC) Sample App Data for AI-based Endoscopy Tool Tracking

The data is automatically downloaded when building the application.

Building and Running H.264 Endoscopy Tool Tracking Application¶

Building and running the application from the top level Holohub directory:

# C++ version
./dev_container build_and_run h264_endoscopy_tool_tracking --language cpp

# Python version
./dev_container build_and_run h264_endoscopy_tool_tracking --language python

Important: on aarch64, applications also need tegra folder mounted inside the container and the LD_LIBRARY_PATH environment variable should be updated to include tegra folder path.

Open and edit the Dockerfile and uncomment line 66:

# Uncomment the following line for aarch64 support
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/aarch64-linux-gnu/tegra/

Enable recording of the output¶

The recording of the output can be enabled by setting record_output flag in the config file <build_dir>/applications/h264/endoscopy_tool_tracking/h264_endoscopy_tool_tracking.yaml to true.

Dev Container¶

To start the the Dev Container, run the following command from the root directory of Holohub:

./dev_container vscode h264

VS Code Launch Profiles¶

C++¶

Use the (gdb) h264_endoscopy_tool_tracking/cpp launch profile to run and debug the C++ application.

Python¶

There are a couple of launch profiles configured for this application:

(debugpy) h264_endoscopy_tool_tracking/python: Launch the h.264 Endoscopy Tool Tracking application with the ability to debug Python code.
(pythoncpp) h264_endoscopy_tool_tracking/python: Launch the h.264 Endoscopy Tool Tracking application with the ability to debug both Python and C++ code.