AI-Generated Cohorts on Federated Data: Practical Guide

AI-generated cohorts are the first practical Sovereign AI workflow that meaningfully shortens the path from research question to feasibility count. A clinical epidemiologist who once spent four to six weeks negotiating SQL with a data engineer per site can now describe a cohort in natural language, watch the system render an OMOP query, dispatch that query to every federated node, and receive site-level summaries within minutes — without a single patient record leaving its host institution.

Why AI-generated cohorts matter now

Three pressures collide in 2026. First, regulatory: the European Health Data Space (EHDS) secondary-use provisions and equivalent national rules now require data custodians to demonstrate that secondary research takes place without exporting identifiable records. Second, scientific: rare-disease trials, post-market surveillance and equity-stratified outcomes work all demand cohorts assembled from multiple custodians at once, which is impossible to do quickly under traditional data-transfer governance. Third, operational: the May 2026 UK Biobank incident — in which approved researchers walked derived data out through a centralised Trusted Research Environment (TRE)’s normal workflow — sharpened the boards of every biobank and ministry of health on what “data minimisation” means in practice.

AI-generated cohort tooling sits exactly where these three pressures meet. It compresses the slowest part of a study (feasibility and recruitment) while preserving the architectural property regulators care about most: data never leaves the source. The trade-off is that the AI layer itself becomes part of the governance surface — the prompt-to-query translation, the airlock review of outputs, and the audit log must be engineered with the same care as the underlying federated infrastructure.

How natural-language prompts become federated queries

Step 1: parse intent against a controlled vocabulary

The researcher types a question. The system does not pass that raw text to an execution engine — it first parses it against the OMOP Common Data Model (CDM) v5.4 vocabulary maintained by the Observational Health Data Sciences and Informatics (OHDSI) consortium. Concepts like “type 2 diabetes”, “GLP-1 therapy” and “HbA1c” resolve to SNOMED-CT, RxNorm and LOINC concept IDs respectively. Ambiguous terms (“recent”, “started on”) are surfaced for human confirmation rather than silently guessed. This grounding step is what turns a language model from a creative writer into a reproducible cohort engineer.

Step 2: render a deterministic OMOP SQL query

Once concepts are pinned, the language model emits SQL — but only SQL that conforms to the OMOP CDM v5.4 table structure (PERSON, CONDITION_OCCURRENCE, DRUG_EXPOSURE, MEASUREMENT, OBSERVATION_PERIOD). Because every federated node hosts an OMOP-conformant view of its source records, the same query runs unchanged across a UK biobank, a Singaporean health system and a Canadian provincial registry. The query is shown back to the researcher in plain English plus raw SQL so a domain expert can sanity-check the logic before dispatch.

Step 3: dispatch to nodes, aggregate at the edge

The query is sent to each participating federated TRE node. Each node executes locally, returns only the pre-agreed result shape (typically a count, a summary table, or a small-cell-suppressed cross-tab) and never exposes row-level data. The orchestration layer combines the per-site responses into a single feasibility report. Researchers see “84,210 individuals meet criteria across 11 sites”; they do not see — and cannot request — the underlying identifiers.

Step 4: automated airlock review of every output

Before any artefact leaves a node, an automated airlock evaluates it against a configurable policy: minimum cell size (commonly five or ten), suppression of small denominators, regex scans for direct identifiers, and statistical disclosure controls on continuous variables. Outputs that pass are released; outputs that fail are quarantined for human review. This turns the architectural promise of federation into an enforceable guarantee.

What changes for the researcher day-to-day

The shift is from procurement to iteration. Under the legacy model, a researcher submits a Data Access Request, waits for sign-off, waits again for a data engineer to write site-specific SQL, and only then sees a number. If the count is too small, the cycle restarts. With AI-generated cohorts on a federated TRE, the same researcher iterates inside an interactive session — try a definition, see the count, refine, repeat. When the researcher commits, the locked definition, the rendered SQL and the language model version are written to an immutable audit log for reproducibility.

Manual cohort discovery vs AI-generated federated cohorts

Governance: what the Sovereign AI layer adds to the threat model

An AI-generated cohort capability is not a free upgrade — and a credible Sovereign AI deployment treats it as a governance event, not a feature release. It introduces three new failure modes. Prompt drift is the risk that two researchers phrasing the same clinical question differently get different cohorts; the OMOP grounding step plus a controlled-vocabulary preview defuses most of this, but the audit log must capture the canonical concept set, not just the prompt. Model substitution is the risk that a future language model version produces different SQL for the same prompt; the audit log pins the model version, and locked cohort definitions are stored as SQL, not as prompts. Output leakage is the risk that an LLM inadvertently surfaces patient detail in a generated narrative; this is contained by routing all numeric outputs through the airlock rather than through the model.

What a working Sovereign AI deployment looks like

The pattern emerging across national biobank programmes globally and leading genomics consortia is a four-layer Sovereign AI stack. Each participating data custodian runs a federated TRE node holding their own source records under their own jurisdiction. Above that sits an OMOP CDM v5.4 conformant view, generated by AI-assisted harmonisation pipelines and validated against the OHDSI Achilles characterisation tool. Above that sits the cohort-construction agent — a language model constrained to emit OMOP SQL, with a controlled-vocabulary preview and a human confirmation step. At the top sits the orchestration and airlock layer, which dispatches queries, aggregates results, and gates every output against a custodian-defined release policy. Federation patents in this space, including US patent 12,519,781, reinforce the principle that compute moves to data rather than the reverse.

Operating the capability — a practical checklist

For a biobank, a ministry of health, or a pharma R&D network preparing to stand up AI-generated cohorts on federated data, the following sequence reflects what works in practice:

1. Confirm OMOP CDM v5.4 conformance at every node before turning on the AI layer; a language model cannot rescue an unharmonised schema.
2. Lock the airlock policy first — minimum cell sizes, suppression thresholds, allowed output shapes — signed off by the custodian’s information governance lead before any researcher prompt is accepted.
3. Pin the language model version in the audit log; treat a model upgrade as a change-management event with regression tests against a benchmark set of cohort definitions.
4. Provide researchers a controlled-vocabulary preview that displays resolved concept IDs before query dispatch, so ambiguity is caught at prompt time.
5. Capture the full audit trail — prompt, resolved concepts, rendered SQL, model version, per-node counts, airlock decisions — queryable by both researcher and oversight committee.
6. Run a small internal pilot before opening the capability to external researchers; the most common early failure is overly permissive airlock policy, not query quality.

Frequently asked questions

Does the AI model see patient records?

No. The language model operates on the cohort definition (the prompt and the OMOP vocabulary), not on the underlying records. Records remain inside each participating federated TRE node. The model emits SQL; the SQL executes at the node; only aggregate outputs that pass the airlock are returned.

Why OMOP CDM v5.4 specifically?

OMOP CDM v5.4, maintained by the OHDSI community, is the de facto common schema for observational health research. It standardises both the table structure and the concept vocabularies (SNOMED-CT for conditions, RxNorm for drugs, LOINC for measurements), which is exactly what an AI cohort builder needs to generate one query that runs unchanged across heterogeneous sites.

How does the airlock decide what to release?

The airlock applies a configurable policy set by the data custodian — typically minimum cell sizes, small-count suppression, statistical disclosure controls on continuous variables, regex scans for free text, and shape checks against the pre-agreed output template. Outputs that satisfy the policy are released automatically; outputs that fail are quarantined for human review.

Can researchers see row-level data after the cohort is defined?

Only if the data custodian’s policy explicitly allows it, and only inside the federated TRE workspace — never exported. The default cohort-construction workflow returns aggregate counts and summary tables. Row-level analysis, where permitted, happens inside the node under the same airlock controls.

What stops a researcher from prompt-engineering around the safety controls?

The model never has the authority to release records — that authority sits with the airlock, which is a deterministic policy engine, not an LLM. A cleverly worded prompt can change which SQL is generated but cannot change what the airlock will let through. The airlock is the load-bearing safety layer.

How is reproducibility preserved when language models change?

Cohort definitions are stored as the rendered OMOP SQL plus the resolved concept set, not as the original prompt. The audit log also pins the language model version used at definition time. To rerun a study, the system replays the locked SQL against the same node versions; the language model is not in the critical path for replication.

Does this approach work across jurisdictions with different data-protection regimes?

Yes — that is the principal architectural reason to use it. Because records stay inside each custodian’s TRE, every participant retains full sovereignty under their own legal regime (GDPR, HIPAA, EHDS, or national equivalents), while researchers receive a single combined feasibility result. The federation layer carries the queries and the aggregate outputs; the records themselves remain under local jurisdiction.