Descriptive Dispatch of Computational Work

Descriptive Dispatch of Computational Work is a method where AI/ML agents receive textual work requests, translate them into specific workload manager commands, and submit them to heterogeneous computing clusters. This approach addresses the challenges of dynamic job transformation and submission in multi-cluster environments, allowing users to provide high-level intent rather than rigid, manager-specific scripts.

Recent research by Vanessa Sochat and Daniel Milroy demonstrates that this agentic dispatch is highly reliable, achieving a 97.9% success rate across 432 experimental runs. The use of descriptive metadata significantly improves execution outcomes, increasing successful job completion from 48% to 87% by eliminating architecture mismatches and boosting performance for certain applications by up to 3.3x. The process involves negotiating resources, selecting the best cluster based on scores, and transforming the job specification for the target environment.

Descriptive Dispatch of Computational Work frames job dispatch as a higher-level orchestration problem rather than a simple submission API call. In this setting, a request for computational work must be interpreted, transformed into a form acceptable to a target workload manager, and submitted reliably. The material emphasizes that this becomes especially difficult when scientific workflows must run across multiple clusters, where schedulers, resource models, authentication mechanisms, queue policies, data locations, and job-description formats can all differ. The central argument is that such dynamic translation and dispatch are well suited to AI/ML-powered agents that can reason about the request, the target environment, and the constraints of the workload manager.

The key contribution is a “descriptive” approach to dispatch, in which work is represented in a richer, more interpretable form that agents can use to generate cluster-specific job definitions and manage the submission process. Instead of relying on static templates or brittle, manually maintained adapters for each backend, the approach allows orchestration agents to handle heterogeneous environments more flexibly. This is particularly relevant for scientific workflows, where jobs may require complex dependencies, specialized runtimes, site-specific configurations, and careful handling of failures or resource availability.

The work matters because it points toward a more resilient model of multi-cluster orchestration. As scientific and engineering workloads increasingly span hybrid HPC, cloud, and institutional cluster environments, the ability to dispatch work automatically and correctly reduces operational overhead, improves reproducibility, and lowers the barrier to using distributed compute resources. More broadly, the paper positions AI/ML agents as practical infrastructure components that can bridge the semantic gap between user-level workflow intent and the low-level requirements of heterogeneous workload managers.