> ## Content Index > Fetch the complete content index at: https://www.notatechguy.com/llms.txt > Use this file to discover other available public pages before exploring further. # BERT-LER: explainable AI reads 75 million health records - URL: https://www.notatechguy.com/bert-ler-explainable-ai-reads-75-million-health-records/ - Published: 2026-08-23T12:38:51.000Z - Updated: 2026-08-23T12:38:51.000Z - Description: BERT-LER, trained on 75 million de-identified patient records, matches benchmark models on clinical prediction tasks while explaining its own reasoning. - Author: Marcello Babbili - Tags: Technology & AI, AI Models A new preprint from researchers tied to Sanofi's data science team describes a transformer model trained on 75 million de-identified patient records that predicts clinical outcomes and shows which parts of a patient's history drove each prediction [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). BERT-LER, posted to arXiv on 20 August, has not been peer-reviewed [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). If the explainability claims survive independent clinical audit, the approach could change how hospitals vet AI before acting on its predictions. The gap between a preprint and a bedside tool is wide, and the stakes are patient lives. **My read:** This is the first EHR foundation-style model I've seen that tries to unify two problems most clinical AI papers treat separately: how you feed lab values into a language model, and how you trace a prediction back to specific data points. The 75-million-patient training scale is serious, and the Sanofi affiliation suggests pharma R&D interest as well as academic curiosity. I don't buy the "many therapeutic areas" generalisation claim yet, because the evidence rests on two benchmarks, one public and one asthma study. And "attributions align with clinically known risk factors" is self-reported, not independently validated. The idea is sound; the proof is thin. ## How BERT-LER turns lab results into language The core problem BERT-LER tackles is representation. Electronic health records are messy: a mix of coded events like diagnoses and procedures, and continuous values like lab results. Most language models handle text tokens well but struggle with raw numbers. BERT-LER, a BERT-style model designed for coded EHR timelines, converts laboratory test results into discrete tokens using percentile-based binning [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). A white blood cell count of 11.2 becomes a token representing "high for this patient population" rather than a raw number the model has to interpret from scratch. The graded information, the actual magnitude, is preserved in the bin assignment. This matters because lab values carry most of the signal in clinical prediction. A model that loses the difference between a slightly high creatinine and a critically high one misses kidney injury. By binning into percentiles, BERT-LER keeps that gradient without requiring the model to learn continuous-value reasoning from scratch. ## The explainability layer On top of the prediction model, the authors pair BERT-LER with Integrated Gradients, a technique that assigns contribution scores to each input token, showing which parts of the EHR sequence pushed the prediction one way or another [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). The authors report that these attributions align with clinically known risk factors [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). Clinical AI faces a harder bar than other domains. A model that explains a wrong prediction could anchor a doctor to a bad decision. The explanation has to be right, not just present. ## What the benchmarks actually show The authors tested BERT-LER on two fronts: the public EHRShot benchmark suite and a real-world asthma severity progression study [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). Across both, they report performance competitive with publicly available benchmark models [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). On laboratory-related tasks specifically, they say BERT-LER often exceeds those benchmarks [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). The distinction matters. "Competitive" on general tasks and "often exceeds" on lab-related tasks is a narrower claim than universal superiority. The authors do not claim it, and the evidence is clear about that boundary. The lab-task advantage, if real, likely flows from the percentile-binning representation: the model is built to handle exactly the data type those tasks depend on. The training scale, 75 million de-identified patients, is the other number that stands out [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). Most public EHR benchmarks work with datasets in the hundreds of thousands to low millions. A 75-million-patient pretraining corpus is an order of magnitude larger, though the dataset is not publicly available and the claim cannot be independently verified from this source. ## What to do about it For a clinical data science team at a hospital or health insurer, the practical signal here is methodological, not a product to buy. BERT-LER is a preprint, not deployed software. But the two ideas, percentile-binning for lab values and Integrated Gradients for token-level attribution, are techniques any team can test on their own data today. Consider a pathology lab that runs predictive models on patient panels to flag likely sepsis cases before full lab results return. The team could bin their own historical lab values into percentiles, train a BERT-style model on the resulting token sequences, and run Integrated Gradients to see whether the model's attributions match the risk factors their clinicians already know: high lactate, rising procalcitonin, falling platelet count. If the attributions line up, the model earns trust. If they don't, the model is learning something wrong, and that is more useful to know than a slightly higher accuracy score. The one thing to check this week: pull up the EHRShot benchmark page, which is public, and read the task definitions. If your organisation's clinical prediction problems map onto those tasks, the benchmark is a free baseline you can compare against. ## What we don't know yet Everything in this story rests on a single non-peer-reviewed preprint [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). The 75-million-patient dataset is not publicly available, so the training scale cannot be independently confirmed. The explainability claims, that attributions align with clinically known risk factors, are author-reported and have not been validated by independent clinicians. The generalisability claim, that the architecture applies to "many therapeutic areas," is supported by exactly two benchmarks [S¹](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com). The corresponding author, Jun Ni Du, is affiliated with Sanofi's Data and Computational Science division [P²](https://arxiv.org/html/2608.20315?ref=notatechguy.com), which raises a question the preprint does not answer: is this research aimed at drug development, clinical deployment, or both? Pharma R&D teams use EHR prediction models for trial recruitment and adverse event detection, as well as bedside care. The next signal: peer review and publication of this work, or its appearance on a public benchmark leaderboard with independently reproducible results. We'll check the claims against whatever comes through first. --- *Sources: [S1 — Explainable Transformer Models for Clinical Prediction Tasks on Struct](https://arxiv.org/abs/2608.20315v1?ref=notatechguy.com) · [P2 — Explainable Transformer Models for Clinical Prediction Tasks on Struct](https://arxiv.org/html/2608.20315?ref=notatechguy.com) · [P3 — microsoft/Debug-XAI](https://github.com/microsoft/Debug-XAI?ref=notatechguy.com) · [P4 — Explainable Transformer Models for Clinical Prediction Tasks on Struct](https://arxiv.org/html/2608.20315v1?ref=notatechguy.com) · [P5 — yhzhu99/ehr-llm-benchmark](https://github.com/yhzhu99/ehr-llm-benchmark?ref=notatechguy.com)* ## More from Not A Tech Guy - [LLM corrections usually die with each session, arXiv preprint says](https://www.notatechguy.com/llm-corrections-usually-die-with-each-session-arxiv-preprint-says/) - [n8n hits 201k GitHub stars but isn't open source](https://www.notatechguy.com/n8n-hits-201k-github-stars-but-isn-t-open-source/) - [Nakama AI agent platform trends on GitHub at 259 stars](https://www.notatechguy.com/nakama-ai-agent-platform-trends-on-github-at-259-stars/) --- *Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.*