The Road to Raw-to-Insights On Active-Scanning Sensors

A novel class of physical AI systems are active-scanning sensors — systems that emit energy into the environment and analyze the reflected signal coming back, measuring distance, structure, velocity, or material properties. Automotive radar, ultrasound scanners, and MRI machines used for industrial and medical imaging are examples of these systems. Like robotics and other physical AI systems, active-scanning sensors have control loops to steer the output based on feedback received from inputs.

Historically, the TX path — responsible for generating waveforms, beam steering, or modulation sequences — was implemented in FPGA logic, where deterministic timing enabled cycle-accurate control of DACs, phased arrays, and RF front-ends.

These are complex systems that require mountains of Verilog code, taking hours to recompile when even small changes are made. And LLM-based AI coding assistants are notoriously bad at enhancing developer productivity for FPGA development.

While developers would love to move these TX algorithms to computing platforms using modern languages like C++ or Python, traditional CPUs introduce massive amounts of jitter and latency that disrupt the timing-critical workload of these pipelines.

GPU Resident Graphs enables these systems to move toward software-defined TX pipelines, where waveform generation and adaptive scanning algorithms are written with CUDA, running directly on the GPU — not in Verilog running on FPGAs.

On the RX side, classical signal-processing algorithms such as beamforming, matched filtering, Doppler processing, and FFT-based spectral analysis processed the returning signal before any higher-level interpretation occurred. These signal-processing techniques leave vast quantities of data on the table. They make too many assumptions about the environment (like an ultrasound probe's speed of sound through tissue), and mask out "noise" that might actually be signal.

Times have changed, though. Jetson Thor is the first low-power SoC to provide more than 2000 AI TOPS of performance, and Blackwell-powered RTX Pro GPUs provide even more firepower. Platforms running Holoscan Sensor Bridge can ingress hundreds of gigabits per second of data with near-zero CPU intervention.

The time is right to bring high-bandwidth real-time raw-to-insights AI inference to these sensor systems.

Holoscan makes it possible to build these systems as software — not hardware. Instead of writing thousands of lines of Verilog and waiting hours for FPGA synthesis, developers can implement real-time sensor pipelines in CUDA and C++, integrate AI models directly into the data path, and iterate at software speed. If you’re building the next generation of radar, ultrasound, robotics, or industrial sensing platforms, download the Holoscan SDK and start building your first Raw-to-Insights pipeline today.