Inference Operator

Authors: Holoscan SDK Team (NVIDIA)
Supported platforms: x86_64, aarch64
Language: Python
Last modified: August 5, 2025
Latest version: 1.1.0
Minimum Holoscan SDK version: 1.0.3
Tested Holoscan SDK versions: 2.2.0, 3.2.0
Contribution metric: Level 2 - Trusted

This operator provides a base class for running inference in medical imaging pipelines.

Overview

The InferenceOperator serves as a foundation for building specialized inference operators, handling model loading, execution, and result management.

Requirements

• Holoscan SDK Python package
• torch (optional, for deep learning models)

Example Usage

from holoscan.core import Fragment
from operators.medical_imaging.inference_operator import InferenceOperator

class MyInferenceOperator(InferenceOperator):
    def __init__(self, fragment, *args, **kwargs):
        super().__init__(fragment, *args, **kwargs)

    def pre_process(self, data, *args, **kwargs):
        # Implement preprocessing logic
        return data

    def predict(self, data, *args, **kwargs):
        # Implement inference logic
        return data

    def post_process(self, data, *args, **kwargs):
        # Implement postprocessing logic
        return data

fragment = Fragment()
inference_op = MyInferenceOperator(
    fragment,
    name="my_inference"  # Optional operator name
)

API Reference

Python

InferenceOperator

Inherits from: Operator

The base operator for operators that perform AI inference.

Methods

Method	Description
__init__(fragment)	Constructor of the operator.
pre_process(data)	Transforms input before being used for predicting on a model.
compute(op_input, op_output, context)	An abstract method that needs to be implemented by the user.
predict(data)	Predicts results using the models(s) with input tensors.
post_process(data)	Transform the prediction results from the model(s).