LLM-Based Multi-Agent Systems for Mathematical Problem Solving: A Comprehensive Literature Review[v1] | Preprints.org

The preprint "LLM-Based Multi-Agent Systems for Mathematical Problem Solving: A Comprehensive Literature Review[v1]" analyzes nineteen multi-agent architectures designed to enhance mathematical reasoning in large language models (LLMs). It highlights benchmarks such as MATH500, GSM8K, AIME, AMC23, GaoKao2023En, Minerva Math, and OlympiadBench as key evaluation datasets for assessing performance on complex mathematical problems, including grade-school arithmetic and competitive-level reasoning tasks.

A central focus of the review is the use of Chain-of-Thought (CoT) prompting, particularly Verifiable Reward-based Prompting (VRP), to improve logical consistency and traceability in reasoning processes. Several systems employ CoT as a foundational technique, with some integrating iterative or multi-turn CoT strategies for deeper decomposition of complex problems.

The review identifies Reinforcement Learning (RL) fine-tuning as a critical training methodology, especially Multi-agent Reinforcement Learning (MARL) approaches like multi-turn Generalized Reward Policy Optimization (GRPO) combined with causal influence and verifiable rewards. These methods aim to mitigate issues such as "lazy agents" and promote role specialization in hierarchical setups.

Hierarchical multi-agent architectures are emphasized as a significant evolution from earlier debate-based systems. These frameworks often feature role-separated agents—such as high-level "Meta-thinking" agents for strategic planning and low-level "Reasoning" agents for execution—coordinated through structured interaction protocols like layered neural optimization or academic review-inspired workflows. For instance, MARS (Multi-Agent Review System) uses an Author-Reviewer-Meta-reviewer structure to reduce computational costs while maintaining accuracy.

The paper also notes persistent challenges, including agent homogeneity when using the same base LLM and lack of standardized computational budgets across studies, which complicate direct performance comparisons. Despite these issues, hierarchical and self-optimizing frameworks represent a promising direction for improving reliability and efficiency in LLM-based mathematical reasoning.

This review provides a systematic examination of Large Language Model (LLM)-based Multi-Agent Systems (MAS) designed to tackle mathematical reasoning tasks. It surveys the current landscape of methodologies, ranging from Chain-of-Thought (CoT) prompting strategies to Reinforcement Learning (RL) fine-tuning techniques, and evaluates their efficacy against standard benchmarks such as GSM8K and MATH500. By categorizing existing approaches, the paper establishes a structured framework for understanding how collaborative agent structures address the inherent limitations of single-model inference in complex logical domains.

A key insight of the work is the analysis of hierarchical multi-agent architectures as a superior paradigm for mathematical problem solving. The review details how these systems decompose intricate problems into manageable sub-tasks, distributing them among specialized agents with distinct roles—such as planners, solvers, and verifiers. This collaborative dynamic enables processes like self-correction, debate, and iterative refinement, significantly enhancing the accuracy and reliability of solutions compared to monolithic LLM approaches.

This research matters because mathematical reasoning remains a critical stress test for the general intelligence capabilities of AI models. By demonstrating that multi-agent frameworks can outperform single models on rigorous benchmarks, the review highlights a scalable path toward more robust and verifiable AI systems. The findings are essential for researchers aiming to develop agents capable of high-level logical deduction and error mitigation, with implications extending beyond mathematics into general scientific reasoning and complex decision-making.

Summary of "LLM-Based Multi-Agent Systems for Mathematical Problem Solving: A Comprehensive Literature Review"

This paper provides a systematic review of large language model (LLM)-based multi-agent systems (MAS) designed for mathematical problem solving, focusing on benchmarks, techniques, and architectural innovations. It evaluates prominent datasets such as MATH500 (competition-level math problems) and GSM8K (grade-school arithmetic), highlighting their role in assessing LLM capabilities in reasoning, symbolic manipulation, and multi-step deduction. The review emphasizes Chain-of-Thought (CoT) prompting as a foundational technique for enhancing interpretability and accuracy, while also exploring reinforcement learning (RL) fine-tuning methods to optimize agent interactions. A key contribution is the analysis of hierarchical multi-agent architectures, where specialized agents (e.g., problem decomposers, solvers, verifiers) collaborate to tackle complex problems through iterative feedback loops. The paper synthesizes recent advances, identifying trends like modular reasoning pipelines and the trade-offs between centralized control and decentralized agent autonomy.

Why it matters: This work is significant for researchers and practitioners in AI-driven mathematical reasoning, as it consolidates fragmented progress in LLM-based MAS and underscores the promise—and challenges—of distributed problem-solving frameworks. By comparing CoT, RL, and hierarchical approaches, the review offers actionable insights for designing next-generation systems that surpass single-model limitations. The emphasis on benchmarks like MATH500 also aligns with the growing need for rigorous evaluation in domains requiring high-precision reasoning, making it a valuable resource for advancing both theoretical understanding and practical applications in AI-assisted mathematics.