1.8 million alerts triaged. Zero known critical misses.
QED built the AI triage layer inside a UK schools safeguarding platform: about 40,000 screen captures a day across 110 schools.
0
Known critical misses
No alert later confirmed critical by the client’s auditors was missed in production. The count is checked against auditor decisions, not graded by the system itself.
1.8M+
Alerts triaged in production
Every decision logged and auditable.
42%
Alerts auto-cleared
Cleared with the reasoning attached. Auditor workload fell 24% at launch.
500K+
Benign alerts cleared before review
Clear non-concerns closed before they reached a human queue.
8 min
Processing SLA
About 40,000 captures a day, triaged against an eight-minute end-to-end SLA.
Client
UK safeguarding software vendor, 110 schools
Duration
12+ months to steady-state operation
Team
4 senior engineers
Engagement
Embedded Partnership
Delivery hub
Zagreb & London
QED ran the engagement as an embedded partnership: four senior engineers inside the client’s team, from the first labelling loop through production and into steady-state operation. In production, QED’s triage layer has processed 1.8 million alerts across 110 UK schools, cleared 42% of them automatically with the reasoning attached, and missed no alert the client’s auditors later confirmed as critical.
The auditing bottleneck
UK schools have a statutory duty to monitor what happens on school devices. Safeguarding software raises an alert when a defined concern appears on screen, and the flags come from keyword matching. Keywords cannot read context: a history essay, a chemistry revision page and a genuine crisis can all trip the same phrase.
At 110 schools and roughly 40,000 captures a day, the queue itself becomes the risk. Auditors wading through benign alerts get slower and start to skim, and a genuine concern sits in the same queue as hundreds of harmless captures: games, revision pages, staff messages.
The brief was blunt: cut the queue without missing a single critical incident. A system that buys tidiness by missing things is useless here; the whole point is the alert it must not miss.
Four passes, ordered by cost
Each alert flows through the pipeline until one stage is confident. Cheap, decisive checks run first, so most alerts never reach the expensive ones. There are two ways out: escalate to a human now, or clear with the reasoning attached.
Anonymise
Names and identifiers replaced before any AI reads the text
Every name, email, address and identifier is swapped out before a language model reads a word. Each person becomes a consistent stand-in so the story still makes sense, public figures are kept for context, and the flagged phrase itself is protected even when the text lifted from the screenshot is garbled. Only after this pass does anything leave for a third-party language model.
Nothing leaves this stage identifiableReason
A language model with a written checklist
The model works through a checklist for each alert: intent, tone, the evasion slang that keyword lists miss. Anything that reads as a critical concern escalates to a human immediately, reasoning attached, and skips the rest of the pipeline.
Critical concern: escalates nowLook
An image model that reads the screenshot itself
A vision model re-examines the screenshot. Text sees the word “crossbow”; vision sees whether the page is a history lesson or a shop listing. It catches what the extracted text alone cannot.
Visual hit: flagged for reviewSweep
A final rules-based pass
A plain list of high-risk phrases and a strict second read cover everything the earlier stages cleared, so a phrase on that list escalates even if every stage above waved it through.
Everything else: cleared with the reasoning attachedThe calls that carried risk
A release that misses one critical case never ships
Before any update goes live, it replays thousands of past, auditor-verified alerts through the real pipeline. One missed critical case blocks the release automatically. That gate holds on the test record rather than the live stream, so production is watched the same way: every auditor decision feeds the next round of tests. The traded cost is deliberate: the system would rather over-flag than under-flag.
Privacy that keeps the signal
The pipeline runs in Google Cloud’s London region. Report images sit behind short-lived links with no long-term copies, and what leaves for a third-party language model has already been anonymised. The anonymiser is built to preserve the evidence the classifier needs.
A deterministic floor under the AI
Underneath the models sits a plain list of high-risk phrases that escalates on its own. If the AI pipeline degrades, misroutes or goes down entirely, the phrases that matter most still reach a human.
Most captures never touch a GPU
The cheapest pass runs first, so expensive hardware only sees what earlier stages could not settle. Vision inference runs on rented spot capacity, two workers sharing each card, scaled up in seconds on live queue depth. Every AI call is individually costed, so the client can see what each decision costs.
The shape repeats outside safeguarding. Any queue where a false negative is catastrophic and false positives drown the humans reads the same: fraud review, content moderation, clinical triage, compliance alerts. The pattern transfers; the numbers have to be earned the same way.
Anyone promising these numbers without the labelling loop is guessing.
What we would tell you before starting
The accuracy is a product of the data loop, not the models. Every auditor decision is stored, every disagreement gets a written reason, and that record becomes the test suite each release must pass; the first months went into building that loop before any modelling. The false positives that remain are stubborn for a good reason: roughly two thirds are staff legitimately discussing safeguarding cases on monitored school devices, exactly the language the system exists to catch, and a category the client’s own policies require a human to clear.