AI assistance shortened adverse event annotation
A peer-reviewed Nature Cancer study tested an agentic language model that extracts six immune-related adverse-event types from clinical notes for human review.
In a randomized crossover study, 17 research staff reviewed 20 unique notes. Timing analysis included 316 participant-note observations (157 manual, 159 AI-assisted); 24 observations lacking recordings were excluded.
| Measure | Manual | AI-assisted |
|---|---|---|
| Median annotation time | 428 seconds | 242 seconds |
| All six current grades matched reference | 19.4% | 25.9% |
Median time fell 43.5%. The cluster-adjusted reduction was 2.71 minutes (95% CI, 1.53–3.89).
Exact-match accuracy included 340 annotations (170 per arm) and required every current grade to match the expert reference. The odds ratio was 1.45 (95% CI, 1.01–2.08; P = .045). Overall label-level grade accuracy did not significantly change.
All study phases involved one health system, without external validation. Automation bias remains unresolved; patient-outcome benefit was not established.