Output Checking and Statistical Disclosure Control in TREs

Output checking is the process of reviewing every result a researcher wants to take out of a Trusted Research Environment (TRE) — tables, figures, model outputs, code — to confirm it cannot disclose information about any individual in the underlying data. Statistical disclosure control (SDC) is the discipline that supplies the rules and judgement for that review: thresholds, dominance checks, differencing analysis, and treatments such as suppression and rounding. Together they implement Safe Outputs, the final pillar of the Five Safes framework, and they are the last control standing between sensitive data and the outside world.

Why output checking matters now

For years, output checking was the unglamorous corner of TRE operations — a queue, a checklist, and a statistician’s afternoon. Two developments changed its status. The first is the May 2026 UK Biobank incident, in which approved researchers exported participant-level data from a centralised platform through its normal workflow: every upstream control — vetted people, approved projects, a secure setting — held, and the absence of a real output control was the entire failure. The second is regulatory: the European Health Data Space (EHDS) requires that secondary use of personal electronic health data happens inside secure processing environments from which personal data cannot be downloaded, meaning every European Health Data Access Body must now operate output checking at national scale. What was a craft practice is becoming a legal requirement and an architectural component at once.

Demand is compounding the pressure. As federated analysis makes multi-site studies routine, the volume and variety of outputs rises — and machine-learning artefacts, which can memorise training records, join tables and plots in the release queue. Manual review alone does not scale to that mix, which is why the field is moving toward semi-automated checking, exemplified by the DARE UK-funded SACRO (Semi-Automated Checking of Research Outputs) project and its associated tooling.

The mechanics of statistical disclosure control

Rules-based checking: the core tests

Most TREs operate a small set of well-established rules. The threshold rule requires that any published count or the cells underlying any statistic describe at least a minimum number of individuals — commonly ten in UK health TREs — because small cells identify people directly. The dominance rule catches magnitude tables where one contributor accounts for most of a total, so the “aggregate” effectively publishes that contributor’s value. Class-disclosure checks catch cells at 0% or 100% — statements that everyone, or no one, in a visible group has an attribute, which disclose information about every member. Differencing analysis looks across outputs: two individually safe tables whose populations differ by one person subtract to reveal that person, which is why checkers review requests against what the same project has already released, not in isolation.

Principles-based checking: judgement over checklist

Rules alone misfire in both directions — blocking harmless outputs that trip a threshold and passing harmful ones the rules never anticipated. The principles-based approach, developed by Felix Ritchie and colleagues and now standard guidance across UK research services, classifies outputs by inherent risk instead. “Safe statistics” such as regression coefficients and test statistics are low-risk by construction and can be released quickly under simple conditions; “unsafe statistics” such as frequency tables, maxima and minima, and percentiles carry inherent disclosure risk and get proportionate scrutiny. The practical effect is faster review of most outputs and deeper review of the few that warrant it — precisely the triage a scaling service needs.

The hard cases: models, code, and derived data

Modern release queues include artefacts the classical literature never contemplated. A trained model’s weights can memorise rare training examples, and membership-inference attacks can test whether a specific person’s record was used in training — so model export is a disclosure decision, not a formality, and may warrant privacy-preserving training or restriction to prediction-only access. Code leaving the environment can embed data in comments, test fixtures, or hard-coded values. Even innocuous-looking intermediate files — a “summary” with one row per patient — are record-level data wearing an aggregate’s name. A mature Safe Outputs function defines, in advance, which artefact types are releasable under which tests, rather than negotiating each novel case from scratch.

Output checking in a Federated Trusted Research Environment

Architecture determines how much weight output checking must bear. In a centralised TRE, checking is one control among several protecting a copied central dataset. In a Federated Trusted Research Environment, it becomes the defining boundary: the data never leaves the source, analysis travels to each custodian, and the only thing that ever crosses the perimeter is an output. That concentration is a strength — there is exactly one gate to build well — and Lifebit’s implementation of it is the automated airlock: every export request is intercepted, screened against SDC rules automatically, and either released, blocked, or escalated to a human checker with the risky elements flagged. Federation adds one further subtlety: in multi-site studies, each custodian’s environment applies its own output checks before site-level results ever combine, so a cross-border analysis clears the strictest applicable standard at every source. The audit trail — who requested what, what the checks found, who approved release — is generated as a by-product, which is exactly the evidence the EHDS’s secure processing environment provisions and any Five Safes assessment ask for.

Common SDC rules at a glance

One dimension the table cannot show is turnaround, and it matters more than operators tend to admit. Output checking is the point where governance is most visible to researchers: a release queue measured in weeks teaches people to batch requests, pad them with everything they might conceivably need, and treat the checkers as adversaries — behaviour that makes every individual review harder and the service less safe. Automating the mechanical tests inverts the dynamic. When the routine majority of outputs clears an automated airlock in minutes and only flagged items wait for a human, researchers submit small, frequent, well-formed requests, and a Federated Trusted Research Environment can hold its strict rules without paying for them in researcher goodwill. Speed, in this corner of governance, is a safety property.

Real-world practice: national services

The direction of travel is visible in production systems. OpenSAFELY, the analytics platform operating over English National Health Service (NHS) primary-care records, applies automatic rounding and suppression to outputs alongside human review, and publishes its disclosure rules openly. The Office for National Statistics’ Secure Research Service — the birthplace of the Five Safes — has long run trained-checker review under principles-based guidance. And the SACRO project brought TRE operators across the UK together to standardise semi-automated checking so that researchers get faster releases and checkers spend their judgement where it matters. National genomics programmes apply the same pattern inside governed research environments: Genomics England, which works with Lifebit on federated analysis of national genomic data, releases only checked aggregate results from its research environment — the model the rest of the sector is converging on.

Common pitfalls

Output-checking programmes fail in predictable ways. Checking in isolation: reviewing each output on its own while ignoring differencing across a project’s release history. Queue collapse: applying identical heavy-touch review to every artefact until turnaround stretches to weeks and researchers batch enormous, harder-to-check requests — the opposite of safety. Unclassified artefact types: having no policy for models, notebooks, or images until the first request forces an improvised decision. Checker burnout and inconsistency: without written standards, two checkers reach different decisions on the same table, and researchers learn to route around the strict one. And the quiet failure of all: treating output control as a manual process bolted onto an architecture that still permits downloads — policy where architecture is needed, the exact gap the UK Biobank incident exposed. Each of these is mitigated by the same combination: automation for the routine, documented principles for the judgement calls, and a TRE architecture in which the checked gate is the only way out.

What to do next

For a TRE operator, a useful self-assessment has four questions. Is there any path out of your environment that bypasses output checking — including screenshots of record-level screens, model exports, and support channels? Are your rules written, published to researchers, and applied consistently — ideally aligned with principles-based classification so safe statistics move fast? Does your tooling automate the mechanical checks and log every decision, so evidence for auditors and access bodies is a query rather than a project? And does your architecture make the gate structural — as in a Federated Trusted Research Environment, where outputs are the only traffic — or does it depend on everyone continuing to follow the rules? Programmes that can answer all four confidently have turned Safe Outputs from a checklist item into what it should be: the enforceable promise that lets custodians, and the public, say yes to research.

Frequently asked questions

What is output checking in a TRE?

Output checking is the review of every result a researcher requests to export from a Trusted Research Environment — tables, figures, models, code — to confirm it cannot reveal information about any individual, applying statistical disclosure control rules and expert judgement before release.

What is statistical disclosure control?

Statistical disclosure control (SDC) is the set of methods for assessing and reducing the risk that published statistics reveal individuals — including threshold and dominance rules, differencing analysis, suppression, rounding, and perturbation.

What is the threshold rule?

The threshold rule requires every published count, and the cells underlying other statistics, to describe at least a minimum number of individuals — commonly ten in UK health TREs — because small cells can identify specific people.

What is principles-based output checking?

An approach, standard in UK research services, that classifies statistics by inherent disclosure risk: low-risk “safe statistics” such as regression coefficients are released quickly, while inherently risky outputs such as frequency tables and extremes receive proportionate human scrutiny.

Can output checking be automated?

Substantially. Automated airlocks apply threshold, dominance, and rounding checks to every export and escalate only flagged items to human checkers; the DARE UK SACRO project standardised this semi-automated model across UK TREs. Judgement-heavy cases, such as model releases, retain a human decision.

How does output checking relate to the Five Safes?

It implements Safe Outputs, the final pillar of the Five Safes framework: even with vetted people, approved projects, secure settings, and de-identified data, results must still be checked so nothing leaving the environment can disclose an individual.

Why is output checking more important in a federated TRE?

Because outputs are the only thing that ever crosses the boundary. In a federated architecture the data never moves, so the checked release gate — the airlock — is the single, well-instrumented point where disclosure risk is controlled, at every participating site.