Monday, August 24, 2026
A hotel support agent receives a simple request: add a quiet-room preference to an existing booking. It asks for the reservation information, checks the record, completes the conversation, and tells the customer the preference has been added. From the transcript, the interaction looks finished. The problem appears only when somebody checks the backend: the booking was never changed. The agent completed the conversation without completing the work. [1], [2]
That example sits near the center of ThinkingBox, a new benchmark and sandbox for testing agents inside stateful business workflows. The researchers built 507 policy-conditioned tasks across retail, hospitality, insurance, banking, and IT and HR support, then ran twelve proprietary and open-weight models through each task twenty times. Instead of grading success primarily from the response or a prescribed tool-call sequence, ThinkingBox inspects the persistent state the agent leaves behind and checks for required, missing, wrong, and collateral effects. [1], [2]
The headline result is uncomfortable in exactly the right way. GPT-5.4, the strongest system evaluated, achieved a 65.36 percent pass@1 score. Across twenty attempts, it found at least one successful trajectory on 91.12 percent of the tasks. Require the same system to succeed on all twenty attempts, however, and the number drops to 25.25 percent. It passed all twenty runs on 128 of the 507 tasks and failed all twenty attempts on 45. Those numbers describe three different properties. We keep pretending they describe one. [2]
Most benchmark reporting compresses performance into a number that answers some version of “can the model do this?” ThinkingBox asks what happens when that capability has to survive repeated execution against state. Its pass@20 measure asks whether at least one of twenty attempts finds a valid solution. Its pass^20 measure asks whether all twenty attempts succeed. The first is useful evidence that a successful trajectory exists. The second starts approaching the question an operator actually cares about: whether the system can reliably reproduce the right outcome when the task arrives again. [1], [2]
The benchmark is deliberately stateful. Each attempt begins from a controlled initial condition, agents interact with simulated users and MCP-compatible tools, and evaluation happens against the records left behind. The evaluator can require a booking change, a support-ticket update, an insurance record, an access modification, or another persistent effect while also rejecting unintended changes. Different valid trajectories can pass because ThinkingBox generally grades the resulting state rather than forcing the model to imitate one canonical sequence of tool calls. [2]
That design exposes failures that conventional monitoring can make look remarkably healthy. Microsoft’s analysis attributes 77.5 percent of failed traces primarily to tool usage, including unsuccessful lookups, failed preconditions, tool errors, and failures to recover effectively. Another 12.1 percent involved a mutating tool that succeeded but changed the wrong entity, value, policy, or side effect. Only 7.9 percent of failed traces were primarily final-response-quality failures, while 2.5 percent involved agents that performed lookups but never made the required state change. [2]
The paper makes the more general point directly. Many failed trials terminated cleanly and included valid state-changing actions. A reasonable-looking response, a successful API call, and the absence of an exception therefore cannot be treated as reliable proxies for completed work. [1] That should bother anyone building agents for real operations more than another model leaderboard.
There is a category error hiding in the way we discuss agent capability. A system demonstrates that it can complete a workflow, and we start speaking as though we have learned that it can operate the workflow. Those are different claims.
The distinction is familiar everywhere else in engineering. A database that commits one transaction correctly has not demonstrated durability. A service that answers one request has not demonstrated availability. A disaster-recovery plan that worked once in staging is not a recovery capability until the organization can reproduce it under controlled conditions. We would consider it absurd to benchmark a production platform by rerunning a failed request nineteen times and then reporting that the platform eventually succeeded.
Agent evaluation has been much more tolerant of that substitution because model capability is probabilistic by construction. pass@k makes sense when the question is whether a model can discover a solution, generate a correct proof, produce working code, or search a space in which multiple attempts are cheap and failures can be discarded. Once an agent is changing bookings, processing refunds, modifying access, updating claims, or touching employee records, attempts stop being free samples from a distribution. They become events with state.
The customer gets one run.
Even when retries are available, the operator has to know whether a retry is safe. A failed lookup can be repeated. A payment may not be. A booking modification may already have succeeded even when the agent failed to understand the response. A second attempt can repair the first failure, duplicate the first success, or create an entirely new state the benchmark designer never wanted.
This is why the ThinkingBox distinction between trajectory and terminal state matters. A tool call can be syntactically correct, authenticated, authorized, and successful from the tool’s perspective while still producing the wrong business state. The model can then close the conversation fluently and give the user an answer perfectly consistent with what it believes happened. Every local signal can look respectable while the system-level outcome is wrong. That is not primarily a language problem. It is an operations problem.
The Green Tool Call Is Not the Receipt
Agent observability is already drifting toward an easy failure mode. We capture the prompt, the reasoning trace where available, the tool selected, the arguments passed, latency, token usage, tool result, and final answer. That is useful telemetry. It still describes the execution path more readily than it proves the intended consequence.
That gap is part of the broader illegibility problem I develop in The Illegibility Crisis: the visible artifact can look complete while the distribution of understanding, decision, and consequence behind it has gone dark. In an agent workflow, the transcript is the artifact. The terminal state is the receipt. [3]
ThinkingBox’s evaluator has a separate view of backend state for a reason. The agent does not get to grade its own effect. If the model says it changed the booking, the booking has to contain the change. If a task requires a support ticket to close with a particular resolution, that state has to exist. If some unrelated field changed along the way, the fact that the requested field also changed does not magically erase the collateral effect. [2]
That is a control worth stealing. For consequence-bearing agents, the source of truth for success should sit outside the agent’s narration of success. The tool response can contribute evidence. The trace can explain the path. The final answer can communicate the result. None of those should substitute for an independently observable postcondition when one exists.
We already know this pattern from infrastructure. A deployment command returning zero does not prove the service is healthy. The control plane accepting a manifest does not prove the workload became ready. An HTTP 200 from an intermediary does not prove the downstream transaction settled. Operators learned to measure the state they actually care about because eventually every optimistic proxy betrays them. Agents do not get an exemption because the proxy can speak English.
Reliability Begins Where the Demo Ends
The 91.12 percent pass@20 result is impressive if the question is capability discovery. GPT-5.4 could find at least one successful path through almost all of the benchmark’s tasks. That tells us the underlying model and scaffold possess substantial breadth. The 25.25 percent pass^20 result tells us something different: on most of those tasks, the successful behavior was not reproduced in every repeated run. [2] That gap is where production engineering starts: a demo needs the successful trajectory; a production system needs the distribution.
The distinction also changes how model comparisons should be read. Microsoft notes that Claude Opus 4.6 and Kimi-K2.6 have nearly identical pass@1 scores, 37.91 and 37.66 percent respectively. Kimi succeeds at least once on 84.22 percent of tasks compared with Opus at 70.02 percent, suggesting broader discoverable capability. Yet Opus succeeds in all twenty attempts on 13.81 percent of tasks while Kimi reaches only 3.16 percent. One system reaches farther; the other is more repeatable inside a narrower territory. [2] A single leaderboard score can flatten that distinction into nonsense.
The model you choose for exploration may therefore not be the model you choose for execution. The model that eventually finds a correct path may not be the model you trust to perform the same state transition unattended at 3 a.m. The appropriate control may even be to constrain a less repeatable model to planning and require a more deterministic mechanism to execute the consequential mutation. That is architecture, not model fandom.
Twenty Runs Is a Harsh Test. Good.
There is a legitimate counterargument to the benchmark. Requiring twenty consecutive successful runs is intentionally severe. ThinkingBox uses synthetic environments and simulated users rather than live production systems. The tasks are controlled, every attempt starts from a clean baseline, and a benchmark score cannot be converted into a universal prediction that an enterprise agent will fail three quarters of the time. The paper is also a preprint rather than a peer-reviewed final publication. [1], [2]
Those limitations should prevent sloppy extrapolation. They do not rescue the weaker evaluation model. The production environment is harsher than the sandbox, even when engineering controls make the deployed system more reliable than the bare agent. Real tools time out. APIs change. Records contain malformed state. Permissions drift. Users provide contradictory information. Dependencies degrade. Concurrent actors change the same objects. Networks partition at inconvenient moments because apparently infrastructure remains committed to dramatic timing.
More importantly, real production does not offer twenty consequence-free attempts from an identical reset state. The benchmark gives the agent a clean slate each time precisely so researchers can measure reproducibility. A real customer request happens against whatever state survived yesterday.
The harshness of pass^20 is therefore useful if we interpret it correctly. It is not a service-level objective and it is not a predicted incident rate. It is a stress test for a property we have been under-measuring: variance in the agent’s ability to realize the same valid end state repeatedly. The result says the successful path often exists, but existence is not enough.
The Operator Standard
The practical response is not to demand that every agent score 100 percent on twenty repeated executions before it touches production. Different consequence surfaces justify different reliability thresholds. An agent summarizing internal documents can tolerate a failure profile that would be reckless for an agent changing bank records, deleting infrastructure, approving refunds, or modifying access.
What should change is the acceptance contract. First, test repeated execution rather than celebrating a single successful trajectory. If the same task produces materially different outcomes from the same starting state, that variance belongs in the deployment decision. A benchmark that can tell you the agent sometimes succeeds has measured capability. An acceptance test has to tell you whether its failure distribution fits the work you intend to delegate.
Second, define success against the state the business actually cares about. Test the required postcondition, prohibited side effects, and any invariants that must survive the action. Use tool traces to diagnose failure, not to declare success merely because the agent chose plausible tools.
Third, design recovery as part of the agent. ThinkingBox’s largest failure category is tool usage, not final-answer fluency. [2] An agent that encounters a failed lookup or rejected operation needs enough state awareness to determine whether it should retry, re-plan, ask the user, escalate, reconcile, or stop. “The tool returned an error” is not a recovery strategy.
Fourth, measure the boundary between intent and effect. Before a consequential operation, know what the agent intends to change. Afterward, independently verify what changed. If the two do not reconcile, do not let a fluent closing sentence convert uncertainty into a completed transaction.
This is the piece agent evaluation has been missing when it treats model performance as though production were a larger benchmark harness. Production is not where we find out whether the agent can succeed. It is where somebody lives with whichever run happened.
ThinkingBox is valuable because it moves the argument away from whether an agent can call tools and toward whether the complete system can reliably perform work. That is a much higher bar, but it is the bar implied the moment we connect a probabilistic actor to persistent state.
The model may know how to do the job. The tool may accept the command. The transcript may look correct. The user may even receive a confident confirmation. None of those observations independently proves that the system left reality in the state it was supposed to leave it. For agents that carry consequence, the benchmark cannot end at capability.
The receipt is the state that survives.
Artifacts are cheap, judgement is scarce.
Per ignem, veritas.
Sources
[1] Z. Li et al., “One Success Isn’t Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows,” arXiv:2608.19741, Aug. 20, 2026. [Online]. Available: ThinkingBox paper on arXiv. [Accessed: Aug. 24, 2026].
[2] L.-C. Tsai, “How we built ThinkingBox to measure whether agents finish the job,” Microsoft Command Line, Aug. 19, 2026. [Online]. Available: Microsoft ThinkingBox technical write-up. [Accessed: Aug. 24, 2026].
[3] P. LaPosta, The Illegibility Crisis: Instrumentation for AI-Era Leadership, 1st ed. Forged Culture, 2025. [Online]. Available: Leanpub. [Accessed: Aug. 24, 2026].