On 16 July, researchers posted an arXiv paper (2607.13396v1) that asks a question every production team should dread: when an AI agent's go-to tool silently stops being the reliable choice mid-session, does the agent notice and switch? [S1] The paper borrows set-shifting from cognitive psychology, the mental flexibility to abandon one strategy and pick up another, and turns it into a behavioural test for LLM agents. What the authors found about how quickly agents lock into routines, and how the framing of a tool list changes their behaviour, raises questions that current benchmarks miss entirely.

The psychology test your agent didn't know it was taking

Set-shifting is a well-established concept in cognitive psychology. It describes the ability to flexibly switch between rules, strategies or mental sets when the task demands change. Humans who struggle with it show rigid, repetitive behaviour, sticking to a strategy that used to work but no longer does. The authors of this paper ask whether LLM agents have the same problem when the tools they rely on quietly change character [S1].

The setup is clever. The benchmark builds tool-skill libraries with built-in redundancies: multiple tools can solve the same task, but they differ in hidden reliability [S1]. One group of tools works well. Then, at a hidden boundary in the session, the reliable group shifts. The tool the agent has been happily using is now the wrong pick. A branched schedule pairs every shift with a no-shift control, so the researchers can separate genuine adaptation from noise [S1].

Accuracy is scored as the joint probability of routing to the correct tool group in every window after a shift [S1]. In plain terms: did the agent switch, and keep switching, every time it should have?

What agents actually do when the ground moves

The authors report that agents, by default, settle into a small recurring routine within a few turns of each boundary [S1]. Call shares, the proportion of attempts directed at each tool, concentrate on a few discrete values after each reliability shift [S1]. The agents find a pattern and stick to it.

When the researchers tested open-weight LLMs in an open-source agentic harness, they found qualitatively distinct failure modes across the same set of routines [S1]. Different models broke in different ways, even when running the same tools and facing the same shifts. The paper also reports that set framing matters: how the toolset presents alternatives, as competing or complementary options, shifts the routing dynamics [S1]. The same tools, the same task, the same reliability shift, but presenting the menu differently changes which tool the agent grabs.

This connects to a wider thread in agent evaluation. Other benchmarks have asked whether agents can handle complex, messy reality. This new paper asks something more unsettling: can an agent even tell when its environment has changed beneath it?

What it means

The core finding is that agents form habits. They find a tool that works, build a routine around it, and keep reaching for it even after it stops being the best choice. Think of a worker who always emails the same colleague for a report, even after that colleague leaves the company.

Set-shifting accuracy, scored as the joint probability of correct routing across every post-shift window, is a stricter measure than a single correct answer [S1]. An agent that switches on the first shift but misses the second scores poorly. This matters because real production environments don't shift once. APIs degrade, rate limits kick in, endpoints move, and fallback services change quality. An agent that adapts to one change but then reverts to its old routine is not reliable in a system that depends on it.

The set-framing result adds another layer. The way you describe tools to an agent, whether you frame them as competing alternatives or complementary pieces, changes which one it picks [S1]. Prompt design and tool descriptions are active levers, not passive documentation. They shape routing behaviour under stress.

What it means for business

A two-person firm running an agent pipeline that calls a search API, a database tool and a summariser knows the pain of tool reliability. APIs go down. Rate limits bite. The fallback service you configured last month might now be worse than the primary. This paper suggests your agent will not necessarily notice.

The practical implication is that agent workflows need explicit reliability monitoring at the infrastructure level and at the agent's own decision layer. If your agent has been calling the same tool for three days and that tool's accuracy has quietly dropped, the agent will likely keep calling it. The paper's findings on set framing mean that how you list and describe tools in your agent's prompt or configuration directly affects whether it will switch under pressure [S1].

For a suburban agency using an agent to pull property data from multiple sources, or a cafe automating inventory checks across two supplier APIs, the risk is the same: the agent finds a routine and sticks. When the routine breaks, the agent may not adapt without an external nudge. Teams building agent systems should consider building in explicit reliability signals, forcing the agent to re-evaluate tool choice periodically rather than trusting it to notice degradation on its own.

What we don't know yet

This is a single-source preprint. All findings are author-reported and have not been peer-reviewed or independently verified [S1]. The abstract does not name specific models, provide numerical accuracy scores, or detail which open-weight LLMs were tested. The paper describes the agentic harness as open-source, but does not explicitly state whether the benchmark itself is open-source [S1].

The paper tests only open-weight LLMs [S1]. Whether proprietary models like those from OpenAI or Anthropic show the same routine-locking behaviour is unknown. No human subjects were included, so there is no direct comparison between agent set-shifting and human cognitive flexibility.

The set-framing finding, that presenting tools as competing or complementary changes routing, is reported as a qualitative observation. Whether it holds across larger tool libraries, different task domains, or longer sessions remains an open question. The next thing to watch is whether the authors release the benchmark code and model-specific results, which would let other teams test their own agents against the same hidden shifts.

If this kind of testing matters to your work, subscribe for more coverage of agent evaluation and reliability research.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.