Meta lets a privacy rule enforce only after it survives shadow mode, and has published how it would take one back.
A column called age, somewhere in Meta's data warehouse, can mean two quite different things depending on what wrote it. Attached to a person it is personal data, to be retained, restricted and kept away from the wrong purposes. Attached to a caching pipeline it is a time-to-live setting that nobody needs to protect. Meta's privacy controls enforce whatever the classifier decides the column is, so a wrong answer either locks down a whole pipeline for nothing or leaves personal data outside the controls built for it.
In June two Meta engineers, Rituraj Kirti and Vasileios Lakafosis, published how the company now makes that call across millions of data assets. Most decisions come from deterministic rules, fixed and versioned conditions that give the same answer every time and can be replayed for an audit. About 85 per cent of traffic resolves that way in milliseconds. The rest, the novel and ambiguous assets, goes to a language model that is slower and costs roughly 400 times as much compute. Both paths return the same result format, carrying the matched rule and the versions used, so enforcement systems downstream have no need to tell them apart.
So when the post describes rules taking traffic away from the model, the classification keeps exactly the authority it had. Retention, access and purpose controls act on the label whichever path produced it. What changes is the machinery producing the label, and what I wanted to know is what that machinery has to show before it is trusted with more of the traffic.
Candidate rules are mined from patterns in decisions people have already reviewed, and every candidate passes the same sequence before it can serve. It is tested against a held-out set of reviewed labels. It then runs in shadow mode, which means it sees production-like traffic and logs what it would have decided while changing nothing. Promotion needs quality, coverage and regression checks to clear, and any rule that could change how sensitive data is protected needs a person's approval first. The thresholds the post does give, 80 per cent purity for single-field rules and 95 per cent for composite ones, only decide which candidates get proposed. The bar for promotion is described as higher and left unquantified.
One broad rule shows the gate doing its job. Promoted, it would have raised coverage, the share of traffic the rules handle without the model, which is the number the whole design exists to push up. In shadow mode its recall on a sensitive class fell, so it would have let through more assets that needed protection. Because the team tracked coverage and recall side by side, they saw the trade and narrowed the rule before it ever served. Meta gives no figure for the drop, or for how far the rule was cut back.
A second case sits at the same point, before anything reached production. An optimisation run scored against the model's own earlier labels appeared to improve, and when it was checked against the frozen set of human-reviewed labels it had drifted away from what the policy intended. The post also describes the safeguards that stop such a run from shipping, without saying whether they came before or after it. Model output never becomes reference truth automatically, and reviewed labels are append-only with their history kept.
The audit side is shown working too. In one review the team took a single past classification, replayed it from its stored trace with the rule, prompt and context versions pinned, and reconstructed why the asset got its label without running the model again.
Taking authority back is in the post as design. A failed candidate is blacklisted for a bounded period, a serving rule is to be retired or revised when its quality falls, and a nightly process samples decisions already served and checks them against reviewed labels. A separate tuning controller can halt the optimisation loop and hand it to a person, though nothing says it reaches a rule that is already enforcing. The two examples Meta uses for the gate both stop a rule before it serves, and this post does not show a live rule being withdrawn.
Its one reported production fault is of a different kind. A bug in how masked context was handled made rules silently fail to match, so the language model took traffic the deterministic path was meant to handle, and coverage seemed to level off while the rule set grew. Fixing it raised coverage and cut model calls. The post doesn't say how the bug was found, which matters for whether the next quiet failure of the intended path would read as a fault or as a plateau.
Before relying on a withdrawal path like this, I would want one deliberate test in a controlled environment, with a bad rule planted on purpose. The trigger should fire on the evidence it was built to watch. Then the rule is disabled at the published pointer, so it stops classifying anything new. Then, as a separate step, the decisions it had already made should be found and corrected, with enforcement on those assets checked on the assets themselves rather than in the classifier's logs. Switching a rule off and repairing what it did are two operations, and a test that checks only the first leaves the second assumed.
What Meta has published, in enough detail to copy, is a gate that refused authority to a rule whose evidence did not hold up. That is worth having on its own. The piece I would ask them for next is a withdrawal shown undoing what a bad rule already did.
Sources
- Rituraj Kirti and Vasileios Lakafosis, Privacy-Aware Infrastructure in the AI-Native Era: An Asset Classification Case Study, Engineering at Meta, 25 June 2026
This piece rests on one self-reported engineering post by the team that built the system. Its only corroboration is an internal comparison by another Meta team, reported in the same post. The post gives no recall or precision figures for the narrowed rule and no numeric promotion bar. The prose was drafted with AI assistance. The research, verification and editorial decisions are the author's.