IMM-based Multiple Object Tracking using a State Prediction Neural Network

The PR-IMM method, proposed by Lim, Paek, and Kong in a September 2026 arXiv paper, integrates a Transformer-based State Prediction Neural Network into the Interacting Multiple Model (IMM) framework to enhance radar-based object tracking. By incorporating radar Doppler measurements to predict object displacement, the data-driven PR mode operates alongside traditional physics-based models (Constant Velocity, Constant Acceleration, and Constant Turn), allowing the tracker to dynamically combine probabilistic outputs for superior handling of nonlinear motion.

Experimental results on the RadarScenes dataset demonstrate that PR-IMM significantly outperforms both standalone data-driven predictors and conventional IMM baselines, achieving a 57.3% reduction in position-estimation error and a 25.3% reduction in ID switches. This hybrid approach effectively preserves the stability and interpretability of Kalman-filter-based motion models while leveraging deep learning’s ability to capture complex, adaptive motion patterns in adverse weather conditions.

The paper introduces PR-IMM, a data-driven multiple object tracking framework for radar-based perception in autonomous vehicles. It builds on the interacting multiple model (IMM) paradigm, which is widely used for tracking maneuvering targets by maintaining and combining several kinematic motion models, and augments it with a state prediction neural network. The central idea is to use learned dynamics to better represent nonlinear object motion than conventional model-based filters, while retaining the probabilistic structure and stability properties of IMM-based tracking.

A key contribution is the hybrid design that couples classical filter-based reasoning with neural-network state prediction. Rather than treating the tracker as a purely black-box learner, PR-IMM embeds the learned predictor within a structured tracking architecture, allowing it to adapt to complex driving maneuvers such as lane changes, sharp turns, and non-rigid motion patterns. This is important because radar measurements for autonomous vehicles are often sparse, noisy, and affected by multipath or clutter, so the tracker must balance flexibility in motion modeling with reliable, stable state estimation.

The work matters because robust multiple object tracking is a critical component of safety-critical perception stacks in autonomous vehicles. By improving nonlinear motion representation while preserving stability, PR-IMM offers a practical path toward trackers that are both more expressive than traditional filters and more dependable than end-to-end learned approaches. The result is a method that can potentially reduce missed detections and track fragmentation in challenging traffic scenarios, supporting more reliable object state estimation for downstream planning and control.