Clinical translation quality dashboard

12 discharge and medication instructions translated from English into Spanish, Polish and Portuguese by three simulated machine translation systems, checked automatically for the mistakes that hurt patients: dropped negations, wrong doses and units, false friends, and untranslated or missing content.

Systems

Average quality score (100 = no issues; each critical issue costs 25, major 5, minor 1). FAIL means at least one critical issue.

Reference

100.0

Human reference translation

36 pass · 0 review · 0 fail

System A

99.6

Fluent, occasionally drops or adds a clause

31 pass · 5 review · 0 fail

System B

84.7

Fast, mishandles negation, numbers and units

14 pass · 0 review · 22 fail

System C

94.9

Literal, uses false friends and leaves English words

15 pass · 17 review · 4 fail

Score by language

SpanishPolishPortuguese
5075100Reference, Spanish: 100.0100Reference, Polish: 100.0100Reference, Portuguese: 100.0100ReferenceSystem A, Spanish: 99.9100System A, Polish: 99.499System A, Portuguese: 99.499System ASystem B, Spanish: 87.588System B, Polish: 81.281System B, Portuguese: 85.485System BSystem C, Spanish: 94.094System C, Polish: 96.897System C, Portuguese: 93.994System C

How good are the checks?

Errors were planted on purpose, so every check can be scored against a known answer.

100%critical errors caught
98%all errors caught
0%false alarms on 95 clean
Added content 12/12
Wrong number critical11/11
Dropped negation critical6/6
False friend critical4/4
Omitted content 2/3
Wrong unit critical5/5
Left in English 8/8

Side by side