CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation

Large language models (LLMs) have demonstrated strong performance in general programming tasks but remain uncompetitive with compiler-based systems such as torch.compile when generating high-performance CUDA kernels, which are essential for modern deep learning workloads . This performance gap stems from the specialized nature of GPU kernel optimization, which requires deep expertise in GPU microarchitecture and computational performance engineering—knowledge not effectively captured by standard LLM training paradigms .

Existing approaches to improve CUDA code generation fall into two main categories: training-free refinement methods and fine-tuning within fixed multi-turn execution-feedback loops. However, both paradigms fail to fundamentally enhance the model’s intrinsic CUDA optimization capabilities, leading to limited performance gains . Training-free methods rely on hand-crafted heuristics and external tools for iterative refinement but do not improve the base model's underlying skills . Similarly, fixed-loop fine-tuning approaches constrain the agent’s autonomy by wasting context on redundant historical data and limiting its ability to learn debugging, search, and profiling strategies .

These limitations highlight the need for more advanced frameworks that go beyond superficial refinement and instead develop true CUDA kernel expertise within LLMs. The scarcity of expert-level CUDA kernel data further complicates supervised fine-tuning, necessitating alternative learning strategies such as reinforcement learning (RL) to enable scalable and effective training .

To address these challenges, CUDA Agent introduces a large-scale agentic RL system designed to systematically improve an LLM’s CUDA optimization abilities through three core components: a scalable data synthesis pipeline, a skill-augmented development environment with automated verification and profiling, and algorithmic RL techniques that ensure stable training . By enabling iterative, multi-turn interactions with execution feedback—including compilation errors, runtime results, and performance profiling—the agent learns to diagnose bottlenecks, apply hardware-specific optimizations, and refine kernels over time, surpassing both static compilers and proprietary LLMs .

This shift toward agentic, long-horizon workflows represents a significant advancement over prior methods, demonstrating that LLMs can achieve state-of-the-art performance in GPU kernel generation when equipped with the right training framework and feedback mechanisms .

CUDA Agent addresses the critical performance gap between Large Language Models (LLMs) and traditional compilers in generating optimized GPU kernels. While LLMs have demonstrated proficiency in general-purpose coding, they consistently struggle to match the efficiency of hand-tuned or compiler-generated CUDA code due to the complex, hardware-specific nature of GPU optimization. This paper identifies that existing self-refinement methods—where models iteratively critique and fix their own code—fail fundamentally because they lack the necessary grounding in hardware performance metrics. To overcome this, the authors introduce CUDA Agent, a large-scale agentic framework that formulates kernel generation as a Reinforcement Learning (RL) problem, treating the optimization process as a sequential decision-making task rather than a one-shot generation.

The key contribution of this work is the integration of an RL-driven search loop with an LLM agent, allowing the system to explore the vast combinatorial space of optimization strategies more effectively than static generation methods. By using actual kernel execution latency as a reward signal, the agent interacts with the hardware environment to validate code changes, learning to navigate the high-dimensional search space of primitives such as thread coarsening, loop unrolling, and memory tiling. The research demonstrates that this agentic approach significantly outperforms standard LLM baselines and, in many cases, exceeds the performance of mature compiler heuristics like NVCC, achieving speedups that validate the efficacy of learning-based search over deterministic rule-based systems.

This research is significant because it bridges the divide between AI-assisted programming and high-performance computing (HPC), offering a solution to the "last mile" optimization problem that LLMs typically fail to solve. By establishing a methodology for LLMs to reason about low-level hardware performance through direct feedback, CUDA Agent automates a process traditionally reserved for expert systems programmers. As computational demands for AI and scientific workloads grow, this agentic approach offers a scalable path toward automating the optimization of software stacks, potentially unlocking substantial performance gains in data centers and supercomputing environments without manual intervention.

# Summary: CUDA Agent – Large-Scale Agentic RL for High-Performance CUDA Kernel Generation

CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation introduces a novel approach to optimizing GPU kernel generation using reinforcement learning (RL) and large language models (LLMs). The paper highlights a critical gap: while compilers excel at low-level optimizations, LLMs struggle with generating high-performance CUDA kernels, and existing refinement methods fail to address fundamental challenges in kernel optimization. The authors propose a CUDA Agent, an agentic RL system that iteratively improves kernel performance by leveraging LLMs for code generation and RL for optimization. This hybrid approach combines the strengths of both methodologies, enabling more efficient and effective kernel tuning than traditional methods.

The paper’s key contributions include: 1. A novel agentic RL framework that integrates LLMs with performance-aware optimization loops, enabling dynamic kernel refinement. 2. Scalable training techniques that leverage distributed RL to handle large-scale CUDA kernel optimization. 3. Empirical validation showing significant performance improvements over baseline methods, such as compiler-based auto-tuning and rule-based optimization.

This work matters because it bridges the gap between high-level code generation and low-level performance optimization, offering a promising direction for automating GPU kernel tuning. By leveraging agentic RL, the approach can adapt to diverse workloads and hardware constraints, making it particularly valuable for HPC, deep learning, and scientific computing applications where kernel efficiency is critical.

Source: [arXiv:2602.24286](https://arxiv.org/abs/2602.24286)