Apple researchers exposed a flaw in private machine learning: the privacy guarantees of batch-sampled DP-SGD break when anyone can observe how many records land in each batch S¹P².
In distributed training, the server aggregating the data, or any party watching network traffic, can see those counts by default S¹. Truncating oversized batches to fit memory limits makes the privacy loss worse still S¹.
DP-SGD, short for Differentially Private Stochastic Gradient Descent, is the standard algorithm for training models on sensitive data. It clips gradients and adds calibrated noise, then processes data in randomly sampled batches. That randomness is what delivers "privacy amplification", a mathematical bonus: because each record might not appear in any given batch, less noise is needed for the same privacy guarantee S¹.
My read: The core insight is sound and the authors carry serious credibility. Feldman, McMillan, and Talwar are established privacy researchers at Apple, not newcomers. What I would watch is whether the empirical advantage over Ganesh's 2025 truncation accounting P⁴ holds outside Poisson sampling, since the paper itself scopes that comparison narrowly. The "easy to implement" claim is author-reported with no third-party engineering review, and the asymptotic efficiency argument for dummy points may rest on assumptions that break in real deployments. This is a preprint, not peer-reviewed S¹, and the mechanism has not been tested in production.
The problem: your batch size is showing
Most sampling schemes that deliver provable privacy amplification, including Poisson sampling (where each record is independently included with a fixed probability) and random allocation (where records are dealt into batches uniformly at random), produce batches of varying sizes, and the privacy proof depends on those sizes staying hidden S¹.
When a batch arrives with 247 records instead of the expected 256, that number alone leaks information about who is in the dataset. Truncation compounds the damage. When memory or compute limits force each batch to a maximum size and an oversized batch gets cut down to fit, the truncation itself can cause a significant jump in privacy loss S¹.
Arun Ganesh published a tighter analysis for truncated Poisson sampling in August 2025 P⁴, giving practitioners a way to account for that loss directly rather than abandoning the privacy amplification bonus.
The fix: padding with dummies
Feldman, McMillan, and Talwar propose adding a random number of dummy points, synthetic entries that pad the batch so its observed size no longer directly reveals the real count S¹. The mechanism maintains the privacy guarantee even after truncation, and the authors provide a general analysis covering both Poisson sampling and random allocation S¹.
The approach also exploits subsampling to amplify the privacy of the count releases themselves. According to the paper, this makes the expected number of dummy points asymptotically smaller than what you would need to directly privatize batch sizes with non-negative noise S¹. Fewer dummy points means less wasted computation on synthetic data that contributes nothing to the model.
For Poisson sampling with truncated batches, the authors report empirically that their mechanism achieves better privacy-efficiency tradeoffs than the direct truncation accounting from Ganesh's 2025 work S¹P⁴. That comparison is scoped to a specific sampling configuration, and no one outside the authors' group has reproduced it.
Who would use it first
The mechanism targets distributed training settings where a central server aggregates gradients from multiple parties, the exact scenario where batch sizes are most exposed S¹. A hospital consortium training a diagnostic model on combined patient records would send batches to a coordination server that can see precisely how many records each institution contributed per round. The dummy-point mechanism would let that server observe a padded count without learning the true batch size.
Implementation in a distributed setting is straightforward, the authors say S¹, though no third party has confirmed this. The analysis supports efficient privacy accounting, meaning the computational cost of certifying the privacy guarantee for each training run should be manageable S¹. The paper does not claim the mechanism works beyond Poisson sampling and random allocation, nor that it eliminates privacy loss from truncation entirely rather than controlling it S¹. It has not been deployed in any production system.
The next checkpoint is peer review, which the authors have not yet entered. Until then, every empirical claim in the paper rests on the preprint alone S¹.
Sources: S1 — Counting on Dummies: Privacy Amplification with Observable Counts and · P2 — Counting on Dummies: Privacy Amplification with Observable Counts and · P3 — wangsw/PrivacyAmplification · P4 — Tighter Privacy Analysis for Truncated Poisson Sampling · P5 — mlpc-ucsd/YOLO-Count
Written from 5 sourced items, 4 of them primary.
