When Faster Decisions Do Not Make Better Agents: 4,500 Task Executions Test How Jev's Savings Transfer in ALFWorld and AndroidWorld

Paper 2610.13092 appeared on arXiv in mid-October 2026 under the title "When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows". Its starting point is practical. Agents finish tasks by making decisions again and again, and LLMs are the usual core decision makers. Jev, which emerged recently, offers to make those decisions faster and cheaper. But it is unclear whether local savings translate into efficient, reliable completion of whole tasks. Our site covered Jev on October 10: TypeSafe AI released it, it is transformer-based but not an LLM, and it outputs probabilities rather than text. The paper calls it a "System One Model" that gives instruction-defined judgments through typed outputs, including probabilities over supplied choices. The design first. The authors pick two representative settings: household embodied tasks in ALFWorld, using the text environment and all 134 tasks in the valid_unseen split, and mobile app use in AndroidWorld, using text-based UI observations and one fixed instance for each of 116 task types. Of five agent families, ALFWorld has ReAct and Reflexion, and AndroidWorld has T3A, AutoDroid and Droidrun's FastAgent. Within each family they replace action selection with one Jev request per decision step, retaining the surrounding observation, history and execution mechanisms, and compare against the original; ALFWorld adds an "LLM-Choice" control where the LLM picks from the same candidate list. Each configuration runs three times on the same tasks, 4,500 task executions in all. All LLM decisions and auxiliary generation use gpt-5.6-luna, and Jev decisions use jev-1.13.0. Cost accounting per million tokens is $0.20 for LLM input, $1.20 for LLM output, $0.042 for Jev input and zero for Jev output.

Now the results. The first layer is effectiveness, and the gap is uneven. On ALFWorld, ReAct-J and Reflexion-J reach 96.02% and 98.01% success, trailing the originals by 1.49 and 1.74 points; with the shared candidate interface the gaps narrow to 1.00 and 0.00 points. On AndroidWorld, Jev trails T3A, AutoDroid and FastAgent by 13.22, 2.30 and 18.10 points, with both AutoDroid variants below 20% success. Difficulty does not depend on sequence length alone: Jev variants hit 100% on ALFWorld's examine, place and cool tasks yet fall behind by about 8.6 and 7.5 points on cleaning, and on AndroidWorld T3A and FastAgent lose 12.5 and 15.3 points even in the shortest-budget stratum. The second layer is cost and time, where savings transfer more readily than speed. On ALFWorld, ReAct and Reflexion cut cost by 50.9% and 32.6% and time by 47.6% and 37.8%. On AndroidWorld, T3A, AutoDroid and FastAgent cut cost by 17.1%, 68.8% and 43.0% but time only by 10.6% and 5.7%, while FastAgent rises 0.7%, and every family uses more tokens per task. The reason is that work outside selection absorbs the savings. Baseline ReAct spends 68.31 of 71.17 seconds selecting actions; Jev cuts that to 17.25 seconds and retains large savings even after adding 17.23 seconds of auxiliary reasoning. T3A's auxiliary generation grows from 67.47 to 96.99 seconds and makes up 81.9% of Jev's inference expenditure; FastAgent makes 44.1% more decisions, since its one-tool-per-step configuration pairs cheaper decisions with longer interaction sequences.

The third layer is the real point: completion determines the value of savings. On matched task-run pairs completed by both configurations, ALFWorld retains 64.8-66.2% lower cost and 59.2-60.8% shorter execution. But count the failures and the picture changes: in Reflexion-J, 2.0% of executions consume 53.3% of inference expenditure through unsuccessful recovery. Per delivered success, cost falls in four families but rises 7.7% for T3A, from 68.70 to 73.97 mUSD, though that interval spans zero; at the observed means, lower completion offsets T3A's 17.1% saving per attempt. FastAgent's nearly unchanged task time and lower completion raise time per delivered success from 239.4 to 377.8 seconds. The fourth layer is failure traces. The authors hand-coded 43 task pairs, 258 executions, with two authors labeling independently and a mean Cohen's kappa of 0.867. In the group where Jev fails persistently while the baseline succeeds, 40 of 57 Jev executions contain failed recovery and 29 contain ineffective search, against 3 and 2 for the baseline. In a typical case, on a Wi-Fi task Jev picks an ambiguously labeled switch and declares completion with the goal unmet, ending any chance of correction; on ALFWorld it selects a handtowel for a cloth and the reasoning then continues on the wrong object. Reflexion-J succeeds on 95.27% of first trials, above the baseline's 93.53%, but then recovers only 11 of 19 initial failures against 25 of 26 for the baseline, so its final success is lower.

The discussion points two ways. For model development, the authors suggest keeping concise numerical outputs while strengthening sensitivity to goals, action prerequisites and contradictory feedback, training on contrasting successful and failed trajectories, and adding a calibrated confidence on whether any available action advances the goal. For system design, treat decision making as an explicit component with observable inputs, outcomes and costs; candidate interfaces should expose target identity, prerequisites and expected state changes; and checks should focus on completion judgments and repeated lack of progress, where failures become terminal or persistent. The authors stress that the comparisons concern the evaluated integrations, since candidate construction, auxiliary generation and feedback all shape results, and cross-family comparisons also reflect differences in interaction budgets and workflow design. Our reading is that the paper rewrites a common marketing story into a testable question: cheap decisions are not cheap outcomes. Its numbers are specific, and so are its limits: text-only observations, five agent families, one fixed seed. The practical advice for engineering teams is that before adopting any "small and fast" decision model, measure how much of your workflow time it actually covers, measure the cost of recovering from failures, and accept it on cost and time per successful delivery, not on the price per call.