Industrial safety
How to scope a PPE detection pilot that proves something
Most safety-technology pilots do not fail; worse, they succeed meaninglessly, run on the three best cameras in perfect light, they produce a number that predicts nothing about the site. This guide is the discipline that makes a pilot worth its weeks: written for the HSE manager who has to defend the decision either way.
Evaluation guide Published 5 min read
01The trap
Why safety-AI pilots usually mislead
The incentives are misaligned from the first meeting. The vendor wants the pilot to succeed, so it proposes the cameras most likely to succeed. The sponsor wants the initiative to succeed, so nobody objects. Twelve weeks later there is a slide with a high detection percentage, measured on unrepresentative views in forgiving conditions, and the rollout it justifies meets the real site: the dusty domes, the backlit gates, the night shift. The gap between the pilot number and the site number becomes the operations team’s problem, and the system’s credibility never recovers.
The fix is not cynicism; it is scoping the pilot as an experiment rather than a demonstration. An experiment can fail, which is exactly what makes its success mean something.
02The shape
The shape of a pilot that produces a decision
-
Step 01
Fix the scope before the vendor does
One site. A defined camera set that includes representative views, not favourable ones: at least one camera in the dust, one against the light, one on the night shift. One or two detection cases with clear definitions, PPE at zone entry is the usual right answer, not five cases thinly.
-
Step 02
Write the success criteria before switching anything on
Three numbers, agreed with the vendor in writing: the detection rate that counts as working, the false-alarm budget per shift the control room can genuinely action, and the response-workflow test, were alerts actually acknowledged and acted on. A pilot without pre-agreed criteria is a demonstration with a start date.
-
Step 03
Establish ground truth, or the numbers are decoration
Someone has to know what actually happened, independently of the system: sampled hours reviewed by a person, counting both error types, events missed and alerts that were wrong. It is tedious, it is the pilot’s real cost, and it is the only source of a number you can defend.
-
Step 04
Run through real conditions, including the tuning
Long enough to cover shift patterns, weather and at least one tuning cycle, because week one’s false-positive rate is not the system’s rate; the question is where it lands after tuning against your conditions, and whether it stays there.
-
Step 05
Decide against the criteria, in writing
Proceed, extend, or stop, by the numbers set in step two. An honest stop is a successful pilot: it spent weeks to save a rollout budget. An extension is legitimate when the criteria narrowly missed and the tuning trajectory is clear. What is not legitimate is moving the goalposts after the fact, in either direction.
Measurement honesty
The two measurement questions that decide everything
Common question
Why insist on measuring false positives as seriously as detections?
Because the failure mode of deployed safety analytics is not missing events, it is being muted. A system tuned to catch everything drowns the control room, gets ignored within weeks, and leaves the site with alerts nobody trusts and a dashboard nobody reads. The alert budget per shift is therefore a first-class success criterion, not an operational footnote.
Common question
What can a pilot legitimately not prove?
That injuries will fall. Injury statistics move slowly and multi-causally; no honest twelve-week pilot demonstrates a TRIR change, and a vendor claiming otherwise is selling. What a pilot can prove: detection and false-alarm performance on your cameras, workflow fit with your control room, and the coverage change in your observation data. The safety-outcome case is built on those, over quarters, by your own numbers.
03Do it properly from day one
Privacy and the workforce, during the pilot
A pilot does not get a governance holiday. Decide before it starts: what footage and records the pilot retains and for how long, who may view them, and how the workforce, contractors included, is informed. Doing this properly in the pilot is not just right; it is realistic, because a rollout will have to, and a pilot that skipped it measured a process that will not exist. The PPE guide’s privacy section covers the design; your legal owner covers the law.
04The one-page version
Before signing a pilot agreement, confirm
- Scope: one site, named cameras including unfavourable views, one or two detection cases
- Success criteria in writing: detection rate, false-alarm budget per shift, workflow test
- Ground truth: who reviews sampled footage, and that both error types are counted
- Duration covers real conditions and at least one tuning cycle
- Alert routing lands in your existing workflow, with named owners per shift
- Retention, access and workforce notification agreed with HSE and legal before day one
- The decision meeting is in the calendar, with proceed / extend / stop defined in advance
Ready to scope one, or want a second opinion on a vendor’s proposal? Interkey scopes pilots on exactly the discipline this page describes, including the parts that make it possible to fail, and will say so if your camera estate is not ready.
05Buyer questions
What buyers ask us
Direct answers to the questions that come up in real evaluations. Anything missing, ask us at the bottom of the page.
How long should a pilot run?
Long enough to cover your real operating variety, shift patterns, weather, at least one tuning cycle, and short enough to keep decision pressure. For most industrial sites that lands in weeks, not days and not a year; the precise duration comes out of scoping, and any vendor quoting one before seeing the site is guessing.
How many cameras should be in scope?
Enough to be representative, few enough to tune honestly: typically a defined handful spanning your view conditions, not the whole estate. The pilot’s job is to predict estate behaviour from a fair sample, and the sample being fair matters more than it being large.
What does a pilot cost?
Its drivers: whether processing runs on temporary on-site hardware, integration depth into your alerting during the pilot, and the ground-truth review effort, which is partly your team’s time. A useful discipline: ask any vendor to price the pilot separately from the rollout, so the decision the pilot produces stays a real one.
From the insights library
Continue exploring industrial safety
PPE detection on the cameras you already own
What is honestly detectable, what your cameras must deliver, and what the tuning weeks involve.
Industry guideAI safety monitoring for oil and gas sites
Running the same pilot on a hydrocarbon site: permits, classified areas and contractor crews.
HubThe industrial safety practice
The practice overview, including the camera-suitability self-check to run before any pilot conversation.
Next step
Scope a pilot that will survive scrutiny
If a pilot is on the table, send the draft scope before signing — reading it against this page is a short conversation that prevents a long disappointment.
Or directly
+966-11-2180999 info@interkey.com.saTawuniya Towers, North Tower, 7th Floor, King Fahad Highway, Olaya, P.O. Box 56835, Riyadh 11564, Saudi Arabia
Elsewhere