Performance metrics
Results reported in the project review, dated 16 November 2026.
Held-out test: 23 incidents. At the High threshold of 0.30, V1 detected 20 incidents. Incident detection and clip precision / recall use different denominators.
- Incident detection
- 87% 20 / 23 incidents · 95% interval: 68–96%
- Clip precision
- 87% 71 true positives / 82 positive clips
- Clip recall
- 82% 71 true positives / 87 labelled positive clips
- False alerts / hour
- 11 Reported 95% interval: 5–20 / hour
Threshold comparison
| Configuration | Threshold | Incidents detected | False alerts / hour | Clip precision | Clip recall |
|---|---|---|---|---|---|
| V1 Standard | 0.854 | 19 / 23 · 83% | 6 | 91% | 78% |
| V1 High | 0.30 | 20 / 23 · 87% | 11 | 87% | 82% |
| V1 Maximum | 0.05 | 21 / 23 · 91% | 19 | 82% | 88% |
| Model B | 0.854 | 11 / 23 · 48% | 14 | 79% | 44% |
High-setting clip counts: 71 true positives, 16 false negatives, 11 false positives and 522 true negatives. The report describes a compact normal test of approximately 0.29 hours.
Broader video evaluation
86 / 100 incidents detected (86%) across a reported 142 unseen videos with approximately 5.2 hours of normal footage. Reported clip precision: 87%; clip recall: 81%. This is a separate evaluation from the 23-incident comparison above.
Historical results and candidate coverage
| Evaluation | Event recall | Normal-footage alerts |
|---|---|---|
| Rules · Snatch 1.0 | 18 / 25 · 72% | 4 / 2.0 h · 2.0 / hour |
| Rules · Snatch-Web | 16 / 25 · 64% | Not paired in the report |
| Fine-tuned verifier · Snatch-Web | 19 / 21 actor-matched · 90.5% | 3 / 0.286 h · 10.5 / hour |
The historical verifier reported clip AUC of 0.96, an event-recall 95% interval of approximately 71–97%, and a false-alert interval of approximately 2–31 / hour. Different protocols and model-selection overlap prevent a controlled model comparison.
Candidate coverage increased from 74% to 92% across 50 labelled events (25 per dataset). Normal-footage candidate generation increased from 3,240 to 3,653 / hour. Coverage measures whether a candidate contains the labelled pair, rather than final detection accuracy.
Source: G024_Report.pdf · Section 5.1, pages 5–6. These are reported experiment results, not live measurements. The report notes that V1 source artefacts were unavailable for independent audit and live-camera performance was unverified.