A new arXiv preprint from UC Berkeley's School of Information argues that language models used to quantify culture have already internalised the very material they're supposed to analyse [S1][P2]. The paper, which has not been peer-reviewed, contends that the entire setup — model, data, annotation, evaluation — doesn't just record cultural reality but actively helps construct it [S1]. If that's right, it breaks something fundamental about how we use AI to understand ourselves. The question is what replaces it.

The thermometer that's also the room

Kent K. Chang, a researcher at UC Berkeley's School of Information, frames the problem with a concept borrowed from quantum physics [P2]. Karen Barad's "agential cut" — the shifting, negotiated line between the object under study and the tool doing the studying — works fine in a lab, where you can tell where the thermometer ends and the liquid begins [S1]. But when a language model measures culture, that line blurs. The model has already ingested enormous quantities of the cultural material it's now being asked to assess [S1]. The instrument is made of the same stuff as the phenomenon.

This isn't a minor methodological quibble. Chang argues that NLP work on culture is a "material-discursive practice" — the tools and choices researchers make don't just observe cultural patterns, they help produce them [S1]. The model you pick, the data you feed it, the annotations you apply, the evaluation metrics you trust: each is a design choice that draws a boundary between what counts as "culture" and what counts as "measurement" [S1]. And that boundary, Chang argues, is entangled from the start [S1].

Three scripts, three warnings

Chang illustrates the argument through three case studies on television and film dialogue, measuring structure, interaction, and deviation [S1]. The specifics of those studies matter less than what they reveal about the apparatus itself. Chang then turns the lens on the tool, examining three aspects of the measurement system: how cultural markers get erased, how models attune — or fail to attune — to historical material, and what happens when agency enters the workflow, meaning the model isn't just measuring but acting [S1].

That last point is particularly sharp. The gap between what a model describes and what it does is widening fast. Chang's paper extends that worry: when an agentic workflow — a model that takes actions, not just answers questions — is part of the measurement apparatus, the line between observing culture and intervening in it doesn't just blur. It vanishes.

What it means

The plain-English version is this: if you use a language model to study culture — to measure how dialogue in films has changed over decades, to track how cultural values shift across communities, to assess whether a TV show represents a particular group accurately — you are not holding up a mirror. You're holding up a lens that was itself shaped by the culture it's looking at, and that lens bends what it sees.

This doesn't make the measurement useless. But it does mean the results come with a built-in circularity that most current NLP work doesn't acknowledge. The model was trained on cultural data; it now measures cultural data; its measurements are shaped by the patterns it already absorbed. Every design choice — which model, which dataset, which annotation scheme, which benchmark — is what Chang calls a "conscious commitment" that is simultaneously methodological and ethical [S1]. You're not just choosing a tool. You're choosing a worldview, and that choice shapes what you find.

A parallel signal comes from researchers at the Alan Turing Institute, the University of Edinburgh, and the University of Copenhagen, whose own position paper argues that "meaning is not a metric" when using LLMs to make cultural context legible at scale [P4]. The two papers approach the same problem from different angles: Chang from critical theory, the Turing Institute team from practical scale. Both arrive at the same warning — treating language model outputs as neutral cultural measurements is a category error.

What it means for business

For a two-person market research firm that uses language models to analyse social media sentiment across cultural groups, the implication is direct. The model you query has already absorbed cultural patterns from its training data — patterns that may over-represent some communities and under-represent others. When you ask it to "measure" cultural attitudes, it's not giving you a clean reading. It's giving you a reading filtered through whatever cultural material it already internalised, plus whatever your prompt, your data, and your evaluation criteria emphasise.

For a streaming platform using NLP to assess whether its original content reflects diverse cultural perspectives, the warning is sharper. If the model erases cultural markers — one of the three failures Chang identifies [S1] — then the "diversity score" it produces may be measuring the model's blind spots, not the content's actual cultural texture. A suburban advertising agency using AI to localise campaigns for different cultural markets faces the same trap: the tool that's supposed to measure cultural fit may be constituting its own version of what "fit" means.

The practical takeaway is not to abandon language models for cultural analysis, but to treat every measurement as a product of a specific apparatus — and to document the apparatus as carefully as the results. Which model. Which data. Which annotations. Which evaluation. Each choice is a commitment, and the reader of the results deserves to know what was committed to.

What we don't know yet

This is a preprint, not a peer-reviewed paper [S1]. Its arguments are conceptual and philosophical, drawing on critical theory rather than presenting new empirical benchmarks [S1]. The case studies on film and television dialogue are illustrative, not comprehensive — we don't know whether the same entanglement problems apply equally to other cultural domains like music, literature, or social media discourse.

We also don't know how the field will respond. Chang proposes a research program that is "theory-driven, empirically rigorous, and culturally contingent" [S1] — but what that looks like in practice, and whether the NLP community will adopt it, remains open. The paper doesn't offer engineering solutions for removing cultural bias from models; it argues that the framing itself needs to change.

The next concrete signal to watch: whether peer review accepts, modifies, or challenges the argument — and whether the growing body of work on AI and cultural measurement, including the Turing Institute's position paper [P4], converges on a shared methodological standard. Until then, anyone using a language model to measure culture should assume the instrument is part of the experiment.

If that's the kind of question worth following, subscribe — we'll be tracking where this lands.

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.