A new arXiv study found that the same reasoning scaffold improved one AI model's strategic thinking by +0.21 points while degrading another by -0.63 [S1]. The gap is not noise. It is a statistically significant crossover that breaks a core assumption about how to make language models reason better, and it raises a question every team deploying AI now needs to answer before swapping models.

The scaffold that cures one model poisons another

The paper, posted to arXiv on 14 July as ID 2607.09743v1, asks whether structured reasoning interventions actually improve strategic economic reasoning in large language models, and whether the answer depends on model architecture [S1]. The researchers used Hotelling's linear city model as their test bed [S1]. Hotelling's model is a classic economics thought experiment: imagine a straight road with customers spread along it, and two vendors choosing where to set up shop. The rational strategy is well known, and the model has been studied for decades, including adaptations for load-balancing problems in computer science [P2]. It makes a clean diagnostic for whether an AI can reason about competitive positioning.

The study evaluated two models: GPT-4.1-mini, a standard instruction-following model, and GPT-5-mini, described by the authors as reasoning-optimized [S1]. Across five conditions (an unscaffolded baseline plus four reasoning interventions), eight questions, three prompt framings, and three repetitions per condition, the team collected 720 individually judged responses [S1].

The headline finding is a crossover interaction. Commitment scaffolding, which asks the model to commit to a strategy before executing it, improved the standard model by +0.21 but degraded the reasoning model by -0.63 [S1]. Principled separation scaffolding, which breaks the problem into distinct reasoning steps, did the opposite: it hurt the standard model by -0.40 and helped the reasoning model by +0.31 [S1]. Both effects were statistically significant (commitment p = 0.040, separation p = 0.002), and the overall crossover interaction was strong (t(7) = 4.79, p = 0.002, d = 1.69) [S1]. The pattern held across seven of eight questions [S1].

The same tool can be medicine for one model and poison for the other.

The gap between knowing and doing

Both models could identify the correct strategy far more often than they could actually execute it [S1]. The authors call this a "declarative-procedural gap": the model knows what it should do but cannot follow through.

Principled separation fully closed this gap for the reasoning model [S1]. No intervention closed it for the standard model [S1].

The researchers also applied adversarial stress-testing, feeding the models hostile or misleading prompt framings. Both models suffered. But the reasoning model took 2.6 times more damage (-1.47 vs -0.57, p = 0.038) [S1]. The harder the baseline question, the less damage the adversarial pressure caused (R-squared = 0.36, p = 0.014) [S1]. On easier problems, the reasoning model had further to fall.

What it means

If you are using prompts or scaffolding techniques to get better strategic reasoning from an AI, the technique that works today may stop working, or actively harm, when you upgrade to a different model. Whether the model is built for instruction-following or for extended reasoning changes which interventions help and which hurt. There is no universal "best prompting strategy." The crossover interaction means the same scaffold can lift one model and drag down another.

This also means the common practice of copying prompt templates from one model to another carries hidden risk. A scaffold that added +0.21 on a standard model could subtract -0.63 on a reasoning model, and you would not know unless you tested.

What it means for business

For a two-person consultancy using AI to draft competitive positioning analysis, or a suburban agency asking a model to reason about where rival businesses might locate, the practical implication is direct. If you have invested time building scaffolding prompts around a standard instruction-following model, those prompts may actively harm performance if you switch to a reasoning-optimized model. The degradation is not marginal. A -0.63 swing on a scoring scale where the baseline is already modest means your carefully structured prompts could make the model worse than no guidance at all.

The adversarial finding matters for any business exposing AI to user inputs. If customers or competitors can phrase prompts in hostile ways, reasoning-optimized models may be more vulnerable, not less. The study found 2.6 times greater degradation under adversarial conditions for the reasoning model [S1]. A cafe using an AI chatbot for customer-facing interactions, or a small firm using AI for negotiation prep, would need to test adversarial scenarios specifically, not assume that a "smarter" model handles them better.

The declarative-procedural gap has a workflow implication. If your AI can tell you the right strategy but cannot reliably execute it, you need a human in the loop for execution, or a different scaffolding approach. Principled separation closed the gap for the reasoning model [S1], which suggests that breaking complex strategic tasks into distinct, separated reasoning steps is worth trying if you are running a reasoning-optimized model.

What we don't know yet

All findings come from the paper's abstract, posted to arXiv on 14 July [S1]. The full methodology, peer-review status, and detailed results have not been independently verified. The study uses Hotelling's linear city model, a theoretical construct from 1929, not real-world market data. The results may not transfer to actual competitive scenarios.

The model name "GPT-5-mini" does not correspond to any widely acknowledged current OpenAI release at the time of writing. The paper describes it as "reasoning-optimized" [S1], but its availability and capabilities are unconfirmed outside this study.

The study tested two models on one economic reasoning task. Whether the crossover interaction generalises to other architectures (Claude, Gemini, open-weight models) or other reasoning domains (mathematical, legal, scientific) is unknown.

The next concrete event: the full paper, if released, would reveal whether the 720 judged responses used blind evaluation, what the four scaffolding interventions were in detail, and whether the adversarial stress-tests reflect realistic attack patterns. Until then, the crossover is a finding worth taking seriously but not yet one to build deployment decisions around.

If you want to catch the next study on AI reasoning gaps before it hits the headlines, subscribe and we will send it straight to your inbox.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.