Introduces OR-Clarify, a benchmark that tests whether LLMs request clarification on incomplete operations-research problem specifications before formulating optimization models.
No benchmark named OR-Clarify exists in the provided search context. While recent research introduces DP-Bench for evaluating LLMs in auto-formulating dynamic programming problems and AskBench for assessing clarification capabilities in general QA, no specific benchmark for interactive optimization clarification is cited.
Current literature does not mention an OR-specific clarification benchmark like OR-Clarify.
Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization addresses a practical failure mode in LLM-assisted operations research: when a problem statement is incomplete or ambiguous, a model may still produce a seemingly valid optimization formulation while silently committing to incorrect assumptions. The paper reframes the task from one-shot model generation to an interactive clarification problem, asking whether an LLM can identify missing or underspecified elements—such as objective priorities, constraint semantics, time horizons, capacity assumptions, or data availability—and request targeted information before writing the mathematical program. Its central contribution is OR-Clarify, a benchmark for evaluating this pre-formulation clarification behavior in OR problem specifications.
The key insight is that correctness in optimization modeling is not only about producing well-formed equations, but also about knowing when the input does not yet contain enough information to do so responsibly. By evaluating clarification-seeking behavior, the work shifts the focus from “Can the LLM formulate a model?” to “Does the LLM know what it does not know, and can it ask the right questions?” This matters because interactive optimization assistants are expected to collaborate with domain experts rather than guess. A system that asks concise, decision-relevant, non-redundant questions can reduce downstream formulation errors, make assumptions explicit, and provide a more auditable path from a natural-language problem statement to a mathematical model. For both practitioners and researchers, OR-Clarify offers a useful diagnostic for building LLMs that are safer and more reliable in interactive OR workflows, where premature optimization can lock in the wrong constraints, objectives, or data interpretations.