A field experiment covering 16,880 work emails across six companies found that GPT-5 rewriting messages in a playful tone made them more emotionally positive, yet did not directly change whether recipients opened or replied [S1]. The engagement effect showed up only through an indirect pathway, buried inside the data. The question is whether the AI tool itself matters, or just the emotional tone it leaves behind on the page.

The experiment that buried its own headline

121 employees at six companies spent three weeks sending work emails under three conditions: writing unaided, having GPT-5 rewrite in a playful tone, and having GPT-5 rewrite in a professional tone [S1]. The design was a randomized crossover, meaning each participant cycled through all three conditions, which helps isolate the effect of the writing method rather than individual style.

The headline result is a null. Neither the playful nor the professional AI rewrite condition directly moved open rates, reply rates, or how fast people responded [S1]. If you stopped at the direct effects, the story would be: AI tone does not matter for engagement.

But the researchers ran a mediation analysis, a statistical method that traces whether a treatment affects an outcome through an intermediate variable. Here, the intermediate variable was emotional positivity, the warmth or lightness of the language itself.

Playful editing increased emotional positivity (B=+0.068, p<0.001). Professional editing decreased it (B=-0.041, p<0.001) [S1]. And within-sender positivity, the relative warmth of a given person's emails compared to their own baseline, strongly predicted both opening (OR=2.05) and replying (OR=3.32, p<0.001) [S1].

So the chain runs: AI editing changes tone, and tone changes behaviour. The AI itself, as a tool, had no direct grip on whether someone opened or replied. The emotional residue of the language did.

What it means

The finding reframes a common assumption about AI writing assistants. Many people adopt them expecting the machine to make their communication more effective by virtue of being AI, as if the model brings a magic quality to the text. This study suggests the opposite. The model's value is narrow and specific: it shifts the emotional register of the words. If it shifts them warmer, engagement can rise. If it shifts them colder, engagement can fall.

The professional tone condition is the cautionary tale. When GPT-5 rewrote emails in a formal, businesslike register, it actually reduced emotional positivity [S1]. The very tone most people would choose for serious workplace communication made the messages less likely to carry the warmth that drives opens and replies.

This connects to older research on email tone. CrowdTone, a 2017 system from Stanford and Microsoft Research, explored how crowd feedback could help people adjust the emotional register of their emails [P4]. The idea that tone shapes how recipients respond is not new. What this study adds is the specific mechanism in an AI-assisted context: the tool does not engage recipients directly. It changes the words, and the words do the work.

GPT-5, the model used in the experiment, was released by OpenAI in August 2025 as a single system that routes between a fast model for everyday queries and a deeper reasoning model for complex problems [P3]. By November 2025, OpenAI had already upgraded to GPT-5.1, describing it as "smarter" and "more conversational" [P2]. The rapid iteration means the specific model in this study may already be one generation behind what most users access today, though the core finding, about tone rather than tool, should carry forward.

What it means for business

For a two-person consulting firm or a suburban real estate agency, the practical takeaway is narrow but real. If you use an AI assistant to draft client emails, the tone setting you choose matters more than whether you used AI at all. A playful rewrite might make your messages warmer, and that warmth, relative to your usual style, could make clients more likely to open and reply.

The risk cuts the other way. Defaulting to a "professional" or "formal" prompt, the setting many people instinctively reach for when writing to clients or senior colleagues, may strip out the emotional warmth that makes people respond. The study found professional editing reduced positivity [S1]. For a cafe sending a newsletter, or a landlord messaging tenants about a repair, a colder tone could mean fewer replies and slower responses.

The indirect nature of the effect also means you cannot measure it by looking at open rates alone and attributing changes to AI. The pathway runs through language quality, which requires a different kind of tracking. A small business owner who switches to AI-assisted emails and sees no immediate jump in replies might wrongly conclude the tool is useless, when the real variable is whether the tone shifted in the right direction.

What we don't know yet

The paper is an arXiv preprint and has not been peer-reviewed [S1]. The causal claims rest on mediation analysis, a statistical technique that traces indirect effects but cannot fully rule out alternative explanations the way a direct experimental result can. The finding that positivity drives opens and replies is partly correlational, even within the experimental framework.

The sample is limited: 121 senders across six companies [S1]. Whether the same indirect pathway holds in larger populations, different industries, or cultures where workplace communication norms differ is an open question. Recipients were not told the emails were AI-assisted, so the study cannot speak to how transparency about AI use might change behaviour.

The next concrete event to watch for is peer review and, if the authors pursue it, replication in a larger or more diverse sample. Until then, the finding is a strong signal, not a settled result.

If this kind of reporting helps you make better decisions, subscribe to keep reading.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.