8 Months of False Positive Data on ESL Students – Complete Update
Reporting back after 8 months of systematic tracking. Posted preliminary findings in April – here’s the complete dataset.
Sample: 847 detection events across my ESL and ELL students at UBC. 3 tools tracked: GPTZero, Turnitin, Originality.ai. I added Proofademic testing for the final 3 months of the study after it came up in forum discussions.
False positive rates on confirmed human ESL writing:
– GPTZero: 28.3% (240/847)
– Turnitin: 19.7% (167/847)
– Originality.ai: 31.2% (264/847)
– Proofademic (3-month subset, n=312): 11.8% (37/312)
For domestic English-speaking students (control group, n=420):
– GPTZero: 11.2%
– Turnitin: 8.9%
– Originality.ai: 13.4%
– Proofademic: 5.4%
The disparity between ESL and domestic students is consistent across all tools. Proofademic showed the smallest absolute disparity and lowest ESL false positive rate of the tools tested.
This data has been submitted to my department and will be part of a policy revision. Will publish more formally eventually.
4 Replies
Join the discussion.
Log In to Reply
eight months of tracking is the kind of evidence that should be in front of policy makers. the anecdotal version of this concern has existed for years. having actual longitudinal false positive data by student population changes the conversation entirely. this should be submitted somewhere beyond a forum thread.
Eight months of systematic data. This is exactly the kind of evidence the policy conversation needs and almost no one is producing. Sharing this with our board policy committee.
The disparity data is the key policy lever. A 28.3% GPTZero false positive rate on ESL students vs 11.2% on domestic students means ESL students are being falsely flagged at 2.5x the rate. This is not a minor calibration issue - it's a systematic bias that affects a specific student population disproportionately.
Any school using these tools without knowing this data is exposing itself to serious equity complaints. The false positive disparity by student population needs to be part of every tool adoption conversation.
Holly's 11.8% vs 28.3% false positive comparison is the most concrete ESL-specific data I've seen anywhere. I'm citing this in a proposal to our dean to pause AI detection use for ESL student submissions pending a proper equity audit. thank you for doing this systematically.