Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?

Resilient strategies in stochastic systems are designed to maintain functionality and performance despite disturbances, such as adversarial attacks or environmental uncertainties, which could otherwise cause decision-flipping or system failure. In multi-agent systems (MAS), resilience is achieved through mechanisms like robust consensus protocols, adaptive recovery, and stochastic stability, ensuring that agents continue to operate effectively even when some fail or are compromised . A strategy profile is considered resilient if it retains its core functionality under partial agent failure and stochastically stable if it maintains low variance in performance metrics, such as visit times to target locations in dynamic environments .

The robustness of a system depends on both its control architecture and communication topology. For instance, resilient consensus in MAS under Byzantine or deception attacks can be achieved if the network topology satisfies a $$2F+1$$ robustness condition, meaning each agent can withstand up to $$F$$ malicious neighbors . When trust edges are present—representing verified, secure communication links—the condition generalizes to $$2F+1$$ generalized robustness with respect to the trust subgraph, enabling reliable state estimation and consensus even under attack . In time-varying topologies, consensus is still possible if a spanning tree exists over sufficiently long intervals, allowing information propagation despite intermittent disruptions .

Stochastic disturbances, such as false-data injection (FDI) or denial-of-service (DoS) attacks, challenge the integrity of AI decision-making. FDI attacks can manipulate prediction errors to arbitrary values by evading detection, undermining system security unless countermeasures like neural network-based detection or distributed filtering are implemented . DoS attacks, which disrupt communication, require control strategies like event-triggered or self-triggered mechanisms to reduce unnecessary transmissions while maintaining consensus, though these introduce bounded errors that trade off with communication efficiency .

In generative AI agents, resilience also involves graceful degradation—maintaining core functionality when advanced features fail—alongside capacity planning for variable loads measured in Requests Per Minute (RPM) and Tokens Per Minute (TPM) . Isolation boundaries and failure injection testing help validate resilience by simulating model inference failures or knowledge inconsistencies before deployment . Adaptive recovery strategies, inspired by chaos engineering, involve deliberate injection of failures (e.g., killing agents or partitioning networks) to harden system responses .

Even with retraining, AI systems remain vulnerable to novel adversarial strategies. For example, human-designed policies have achieved over 97% win rates against the Go-playing AI KataGo by creating board states outside its training distribution, tricking it into catastrophic errors—a demonstration of bounded rationality in AI decision-making . These adversarial strategies exploit the fact that AI systems, despite high performance, operate within algorithmic and data-driven constraints that do not generalize to all possible scenarios .

Thus, the threshold for breaking a "winning" strategy depends not only on the magnitude of disturbance but also on the adaptability, topological robustness, and detection capabilities of the system. Resilience is not absolute but a function of design trade-offs between accuracy, responsiveness, and robustness

This paper investigates the robustness of optimal strategies within stochastic systems—specifically modeled as Markov Decision Processes (MDPs)—when subjected to "decision-flipping disturbances." This form of disturbance involves an adversarial agent or environmental fault arbitrarily altering an agent's selected action after it has been chosen but before it is executed. The authors formalize the concept of resilient strategies, defined as policies that can satisfy specific reachability or safety objectives despite a bounded probability of such action corruptions. The study rigorously analyzes how these disturbances interact with the inherent stochasticity of the environment to degrade system performance.

A key contribution of this work is the quantitative analysis of the "breaking point" for winning strategies. The authors establish theoretical bounds and provide computational methods to determine the maximum level of decision-flipping probability a strategy can tolerate before failing to meet its specification. By characterizing the trade-offs between the probability of disturbance and the feasibility of achieving the desired outcome, the paper offers algorithms for both verifying the resilience of existing policies and synthesizing new ones that are inherently robust against these specific perturbations.

This research is critical for the deployment of AI agents in safety-critical domains where actuation errors or adversarial interference are realistic threats. In complex, real-world stochastic environments—such as autonomous robotics or high-frequency trading systems—the assumption of perfect action execution is often violated. By providing a framework to quantify and guarantee resilience against decision-flipping, this work bridges the gap between theoretical optimality and practical reliability, ensuring that winning strategies remain viable even under duress.

Summary of "Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?"

This paper investigates the robustness of decision-making strategies for AI agents operating in stochastic environments, where small perturbations can "flip" decisions and degrade performance. The authors formalize the problem of strategic resilience by modeling adversarial noise that seeks to disrupt an agent's optimal policy. They introduce a framework to quantify the minimal disturbance required to break a winning strategy—i.e., the "resilience threshold"—and analyze how this threshold depends on factors like noise structure, action space dimensionality, and the agent's decision margin. The work extends classical robustness analysis (e.g., in machine learning) to stochastic control settings, where decisions are made under uncertainty rather than adversarial attacks.

Key contributions include: 1. A theoretical characterization of resilience thresholds for Markov Decision Processes (MDPs) and bandit problems, showing that resilience scales with the agent's decision confidence and the entropy of the action space. 2. Algorithmic insights for designing inherently resilient policies, such as leveraging exploration-exploitation trade-offs or incorporating noise-aware regularization. 3. Empirical validation in synthetic and real-world stochastic domains (e.g., reinforcement learning for robotics), demonstrating that even small perturbations can significantly erode performance unless strategies are explicitly hardened.

Why it matters: This work bridges gaps between stochastic optimization and robustness research, offering tools to evaluate and improve AI systems in noisy, real-world settings. For practitioners, it highlights the fragility of seemingly optimal strategies and provides methods to preemptively harden them. Theoretically, it advances the study of adversarial robustness beyond deterministic settings, addressing a critical challenge for deploying AI in uncertain environments.