Rolling Round-Robin Rate: Standard Heterogeneous Throughput for SPEC CPU

Rolling Round-Robin Rate (RRR) is a new exhibition run style introduced in SPEC CPU 2026 (arXiv:2609.29681) designed to measure standard heterogeneous throughput in multi-copy environments. Unlike traditional homogeneous SPECrate modes, RRR runs a deterministic, staggered schedule where each core executes all benchmarks in the suite sequentially but starting at different points, creating a controlled mix of different application types.

This methodology addresses the gap in benchmarking heterogeneous, multiprogrammed workloads common in modern cloud and server systems, where shared resources like caches and memory bandwidth are contested by diverse jobs. RRR provides a standardized, reproducible way to evaluate system performance under realistic mixed-load conditions, exposing variability and interference that homogeneous runs often hide.

The paper proposes a SPEC-like scoring model for RRR reporting, calculating per-benchmark average throughput and coefficient of variation, then averaging the geometric means of each copy across the suite to form a final suite score. This approach preserves the familiar SPEC throughput tradition while adding richer diagnostics for noisy-neighbor studies, OS scheduling, and resource partitioning research.

Scope and motivation. The paper addresses a gap in SPEC CPU’s throughput methodology: the long-standing SPECrate mode measures homogeneous throughput by running many copies of the same benchmark concurrently, which is useful but does not fully reflect how modern cloud and server systems operate. In production environments, machines commonly host heterogeneous collections of jobs with different resource demands, execution times, and performance sensitivities. The authors argue that a homogeneous multi-copy rate can therefore miss important platform behavior, especially under realistic mixed workloads where contention, scheduling, and resource sharing are central.

Key contribution. The proposed contribution is a standardized heterogeneous-throughput metric for SPEC CPU called the Rolling Round-Robin Rate. Rather than measuring only how many identical benchmark copies can be run at once, the method uses a rotating or rolling round-robin execution of a mixed set of SPEC CPU workloads. The goal is to produce a reproducible rate that captures sustained throughput under a heterogeneous job mix while preserving the comparability and normalization conventions of the SPEC CPU ecosystem. This makes it possible to evaluate how processors, compilers, memory systems, and platform software behave when serving diverse concurrent workloads instead of a single replicated benchmark.

Why it matters. This matters because server-class and cloud performance is increasingly determined by how well systems handle mixed, multi-tenant workloads rather than by peak homogeneous scaling alone. A standard heterogeneous rate can help users distinguish platforms that scale well only under identical workloads from those that maintain efficiency when running varied benchmark classes simultaneously. For capacity planning, cost-performance analysis, and hardware or compiler selection, such a metric offers a more realistic bridge between traditional SPEC CPU results and the heterogeneous throughput demands of modern data-center workloads.