Streaming Server Operators#
Authors: Holoscan Team (NVIDIA)
Supported platforms: x86_64, aarch64
Language: C++, Python
Last modified: November 3, 2025
Latest version: 1.1.0
Minimum Holoscan SDK version: 3.5.0
Tested Holoscan SDK versions: 3.5.0
Contribution metric: Level 1 - Highly Reliable
The video_streaming_server operator provides a modular streaming server implementation with separate upstream, downstream, and resource components. This split architecture allows for better separation of concerns and more flexible streaming pipeline configurations.
📚 Related Documentation: - Main Operators README - Setup, dependencies, NGC downloads, and Python examples - Server Application README - Complete server application with usage examples - Client Operator README - Companion client operator documentation - Testing Documentation - Integration testing and verification
Architecture Overview#
The Streaming Server operators integrate with the Holoscan Server Cloud Streaming library to provide comprehensive multi-client streaming capabilities:
┌─────────────────────────────────────────────────────────────────────────────────┐
│ Holoscan Application │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────────────┐ │
│ │ StreamingServerResource │ │
│ │ │ │
│ │ ┌─────────────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Holoscan Server Cloud Streaming │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────┐ ┌─────────────────────────────────────────────┐ │ │ │
│ │ │ │ StreamingServer │ │ Network Protocol Stack │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ │ • Multi-client │───▶│ • Media Transport & Encoding │ │ │ │
│ │ │ │ Management │ │ • Connection Management │ │ │ │
│ │ │ │ • Frame Routing │ │ • Security & Authentication │ │ │ │
│ │ │ │ • Callbacks │ │ │ │ │ │
│ │ │ └─────────────────┘ └─────────────────────────────────────────────┘ │ │ │
│ │ └─────────────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ └──────────────────────────────────────┼──────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────────────┐ │
│ │ Processing Pipeline │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │
│ │ │StreamingServer │ │ Processing │ │StreamingServer │ │ │
│ │ │UpstreamOp │ │ Operators │ │DownstreamOp │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • Frame Receive │───▶│ • Format Conv │───▶│ • Frame Processing │ │ │
│ │ │ • Client Frames │ │ • AI/ML Ops │ │ • Tensor → VideoFrame │ │ │
│ │ │ • Tensor Output │ │ • Filtering │ │ • Multi-client Send │ │ │
│ │ │ • Validation │ │ • Enhancement │ │ • Optional Processing │ │ │
│ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ │
│ │ │ │ │ │ │
│ └───────────┼───────────────────────┼────────────────────────────┼────────────┘ │
│ │ │ │ │
│ ▼ │ ▲ │
│ ┌─────────────────┐ │ ┌─────────────────────────┐ │
│ │ Input Sources │ │ │ Output Sinks │ │
│ │ │ │ │ │ │
│ │ • Client Frames │ │ │ • Client Connections │ │
│ │ • Network Data │ │ │ • Processed Frames │ │
│ │ • Remote Cams │ │ │ • Multi-cast Streams │ │
│ └─────────────────┘ │ └─────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────┐ │
│ │ Optional Processing │ │
│ │ │ │
│ │ • AI/ML Inference │ │
│ │ • Computer Vision │ │
│ │ • Frame Enhancement │ │
│ │ • Analytics │ │
│ └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────┘
│
┌──────────────┼──────────────┐
│ Network │
│ │
│ ┌──────────────────────────┴──────────────────────┐ │
│ │ Multiple Clients │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌──────────┐│ │
│ │ │ Client 1 │ │ Client 2 │ │ Client N ││ │
│ │ │ │ │ │ │ ││ │
│ │ │ • Holoscan │ │ • Holoscan │ │• Holoscan││ │
│ │ │ Client │ │ Client │ │ Client ││ │
│ │ │ Streaming │ │ Streaming │ │ Streaming││ │
│ │ └─────────────┘ └─────────────┘ └──────────┘│ │
│ └─────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
Component Interactions#
-
Multi-Client Management: The StreamingServerResource manages multiple simultaneous client connections through the Holoscan Server Cloud Streaming library
-
Upstream Processing: StreamingServerUpstreamOp receives frames from connected clients and converts them to Holoscan tensors for processing
-
Pipeline Integration: Frames flow through the standard Holoscan processing pipeline (AI/ML, computer vision, analytics, etc.)
-
Downstream Distribution: StreamingServerDownstreamOp takes processed tensors and distributes them to all connected clients
-
Network Protocol Handling: The cloud streaming library manages:
- Multi-client signaling and negotiation
- Media encoding/decoding and transport
- Load balancing and connection management
-
Security and authentication
-
Bidirectional Communication: Supports simultaneous receiving from clients (upstream) and sending to clients (downstream)
Architecture Components#
holoscan::ops::StreamingServerResource#
A shared resource that manages the underlying StreamingServer instance and provides: - Centralized server lifecycle management - Event handling and callback management - Configuration management for server parameters - Frame sending and receiving coordination between operators
holoscan::ops::StreamingServerUpstreamOp#
An operator that receives video frames from streaming clients and outputs them as Holoscan tensors:
- Receives frames from connected clients via the StreamingServerResource
- Converts received frames to holoscan::Tensor format
- Provides duplicate frame detection to ensure unique frame processing
- Outputs tensors to the Holoscan processing pipeline
holoscan::ops::StreamingServerDownstreamOp#
An operator that receives Holoscan tensors and sends them to streaming clients:
- Takes holoscan::Tensor input from the processing pipeline
- Converts tensors back to video frame format
- Sends processed frames to connected clients via the StreamingServerResource
- Supports optional frame processing (mirroring, rotation, etc.)
Benefits of Split Architecture#
- Modularity: Each component has a single responsibility (resource management, receiving, or sending)
- Flexibility: You can use only upstream, only downstream, or both depending on your pipeline needs
- Shared Resource: Multiple operators can share the same StreamingServerResource instance
- Better Testing: Each component can be tested independently
- Clear Data Flow: Explicit tensor-based input/output ports make data flow obvious
- Processing Integration: Seamless integration with Holoscan's tensor processing pipeline
Parameters#
StreamingServerResource Parameters#
port: Port used for streaming server- type:
uint16_t -
default: 48010
-
is_multi_instance: Allow multiple server instances - type:
bool -
default: false
-
server_name: Name identifier for the server - type:
std::string -
default: "HoloscanStreamingServer"
-
width: Width of the video frames in pixels - type:
uint32_t -
default: 854
-
height: Height of the video frames in pixels - type:
uint32_t -
default: 480
-
fps: Frame rate of the video - type:
uint16_t -
default: 30
-
enable_upstream: Enable upstream (receiving) functionality - type:
bool -
default: true
-
enable_downstream: Enable downstream (sending) functionality - type:
bool - default: true
StreamingServerUpstreamOp Parameters#
width: Frame width (inherits from resource if not specified)- type:
uint32_t -
default: 854
-
height: Frame height (inherits from resource if not specified) - type:
uint32_t -
default: 480
-
fps: Frame rate (inherits from resource if not specified) - type:
uint32_t -
default: 30
-
allocator: Memory allocator for tensor data -
type:
std::shared_ptr<Allocator> -
video_streaming_server_resource: Reference to StreamingServerResource - type:
std::shared_ptr<StreamingServerResource>
StreamingServerDownstreamOp Parameters#
width: Frame width (inherits from resource if not specified)- type:
uint32_t -
default: 854
-
height: Frame height (inherits from resource if not specified) - type:
uint32_t -
default: 480
-
fps: Frame rate (inherits from resource if not specified) - type:
uint32_t -
default: 30
-
enable_processing: Enable frame processing (mirroring, etc.) - type:
bool -
default: false
-
processing_type: Type of processing to apply - type:
std::string - default: "none"
-
options: "none", "mirror", "rotate"
-
allocator: Memory allocator for tensor data -
type:
std::shared_ptr<Allocator> -
video_streaming_server_resource: Reference to StreamingServerResource - type:
std::shared_ptr<StreamingServerResource>
Input/Output Ports#
StreamingServerUpstreamOp Ports#
Output Ports:
- output_frames: Output port for frames received from clients
- type: holoscan::Tensor
StreamingServerDownstreamOp Ports#
Input Ports:
- input_frames: Input port for frames to be sent to clients
- type: holoscan::Tensor
Example Usage#
Complete Pipeline Setup#
// Create allocator resource
auto allocator = make_resource<UnboundedAllocator>("allocator");
// Create the shared StreamingServerResource
auto video_streaming_server_resource = make_resource<ops::StreamingServerResource>(
"video_streaming_server_resource",
Arg("port") = 48010,
Arg("server_name") = "MyStreamingServer",
Arg("width") = 854,
Arg("height") = 480,
Arg("fps") = 30,
Arg("enable_upstream") = true,
Arg("enable_downstream") = true,
Arg("is_multi_instance") = false
);
// Create upstream operator (receives frames from clients)
auto upstream_op = make_operator<ops::StreamingServerUpstreamOp>(
"streaming_upstream",
Arg("allocator") = allocator,
Arg("video_streaming_server_resource") = video_streaming_server_resource
);
// Create downstream operator (sends frames to clients)
auto downstream_op = make_operator<ops::StreamingServerDownstreamOp>(
"streaming_downstream",
Arg("enable_processing") = false,
Arg("processing_type") = "none",
Arg("allocator") = allocator,
Arg("video_streaming_server_resource") = video_streaming_server_resource
);
Requirements & Setup#
For complete setup instructions including: - Holoscan SDK 3.5.0 and CUDA 12.x requirements - NGC binary downloads (server streaming binaries) - Build troubleshooting
See the Main Operators README for detailed setup instructions.
Deployment on NVCF#
The Holoscan cloud streaming stack provides plugins with endpoints required to deploy the server docker container as a streaming function. You can push the container and create/update/deploy the streaming function from the web portal.
Push Container#
Note: You first must docker login to the NGC Container Registry before you can push containers to it: https://docs.nvidia.com/ngc/gpu-cloud/ngc-private-registry-user-guide/index.html#accessing-ngc-registry Tag the container and push it to the container registry:
docker tag simple-streamer:latest {registry}/{org-id}/{container-name}:{version}
docker push {registry}/{org-id}/{container-name}:{version}
For example, if your organization name/id is 0494839893562652 and you want to push a container to the prod container registry using the name my-simple-streamer at version 0.1.0 then run:
docker tag simple-streamer:latest nvcr.io/0494839893562652/my-simple-streamer:0.1.0
docker push nvcr.io/0494839893562652/my-simple-streamer:0.1.0
Set Variables#
All the helper scripts below depend on the following environment variables being set:
# Required variables
export NGC_PERSONAL_API_KEY=<get from https://nvcf.ngc.nvidia.com/functions -> Generate Personal API Key>
export STREAMING_CONTAINER_IMAGE=<registry>/<org-id>/<container-name>:<version>
export STREAMING_FUNCTION_NAME=<my-simple-streamer-function-name>
# Optional variables (shown with default values)
export NGC_DOMAIN=api.ngc.nvidia.com
export NVCF_SERVER=grpc.nvcf.nvidia.com
export STREAMING_SERVER_PORT=49100
export HTTP_SERVER_PORT=8011
Create the Cloud Streaming Function#
Create the streaming function by running the provided script after setting all the required variables:
./nvcf/create_streaming_function.sh
Once the function is created, export the FUNCTION_ID as a variable:
export STREAMING_FUNCTION_ID={my-simple-streamer-function-id}
Update Function#
Update an existing streaming function by running the provided script after setting all the required variables:
./nvcf/update_streaming_function.sh
Deploy Function#
Deploy the streaming function from the web portal: https://nvcf.ngc.nvidia.com/functions
Test Function#
Start the test intermediate haproxy by running the provided script after setting all the required variables:
./nvcf/start_test_intermediate_haproxy.sh
Please note that the test haproxy server should be running on a separate machine, either on the client machine or a separate one.
Note: If the test haproxy is still running, and you wish to test the executable or docker file again you must first stop it:
./nvcf/stop_test_intermediate_haproxy.sh
Python Bindings & Applications#
For Python usage, application examples, and testing: - Main Operators README - Python bindings overview and setup - Server Application README - Complete Python server implementation - Testing Documentation - Integration testing guide
Additional Resources#
For more information on NVCF Cloud functions, please refer to NVIDIA Cloud Functions documentation.