Corvus ISR Publishes Transparent Benchmark of Tracking Models

Corvus ISR tracker benchmark matrix (seed 1337) — The published matrix — every row reproducible. Source: corvusisr.com/benchmark

Corvus ISR has released a comprehensive public tracker benchmark comparing two distinct models in a synthetic, fully controlled environment. This benchmark uses a fixed-seed scene with perfect ground truth, enabling an exact measurement of each model’s tracking performance over a 20-second warm-up followed by a 120-second evaluation period. Such synthetic scenes are critical for avoiding real-world noise, permitting precise assessment of algorithmic strengths and weaknesses.

The benchmark pits a simple greedy nearest-neighbour model (v1) against a more sophisticated confirmed-track auction approach (v2). The v1 model employs two-pass greedy association, constant-velocity prediction, and fixed 2-second coasting—serving as the baseline. In contrast, v2 incorporates advanced features like three-tier auction association, velocity-consistency gating, and noise-scaled reservation pricing, which collectively aim to improve tracking reliability.

The results are striking: in a scenario with 150 moving objects at 2 frames per second, the ID switches per minute dropped from 2,042 to 1,183, representing a 42.1% reduction. When scaled to 400 movers, the improvement remained consistent—14,032 to 8,040 ID switches, a 42.7% decrease. These numbers highlight the significant gains achieved through the newer track confirmation methodology, even under challenging conditions like occlusion, low frame rate, and sensor noise.

It’s important to understand the metric honesty behind these numbers: the ID switch metric counts every change in the assigned identity, including re-acquisitions and fragmentations, making it a stricter measure than typical MOT challenge standards. This rigorous approach ensures that published failure data genuinely reflect tracker performance, not just model success. Both models still generate thousands of errors per minute under stress, emphasizing that no tracking system is perfect.

Corvus ISR live demo — The live demo — press “Run benchmark” to reproduce the numbers. Source: corvusisr.com/demo

The value of sharing these failure metrics lies in transparency: synthetic scenes provide perfect ground truth, so the recorded errors are unequivocal measurements, not marketing hype. Every future iteration of a tracker will be publicly logged against the same seed, fostering a culture of open validation. As Corvus ISR states, “Vendors who show only successes ask for faith; a published failure matrix asks for measurement.”

From an engineering perspective, v2 demonstrates impressive speed, averaging around 1.2 milliseconds per sensor tick with 400 objects—well within real-time constraints. Even in the worst case, with around 5 milliseconds per frame, it remains suitable for live deployment. Curious readers can reproduce it live by clicking the “Run benchmark” button on the demo page, with no sign-up or NDA required. This fully synthetic environment ensures a level playing field for evaluating and understanding tracking performance.

By using perfect ground truth from synthetic scenes, Corvus ISR exemplifies a scientific approach to evaluating tracking algorithms—providing transparent metrics that can be independently verified. This methodology helps push the industry toward more robust, reliable solutions by emphasizing measurement over marketing. Interested in seeing the results firsthand? We invite you to run the benchmark yourself and explore the capabilities of these tracking models in your own experiments.