What Happens When the Detector Is Wrong – A Real Story
Sharing this because I think it needs to be heard before more schools go down this road.
In December we had a grade 11 student flagged by Turnitin for a history essay. Score: 74%. Department head treated it as strong evidence of AI use and called a meeting with the student and parents. The student denied it. Parents requested to see the methodology behind the score. We couldn’t explain it to them in a way that held up. The father started mentioning legal consultation.
We ultimately couldn’t proceed with any formal finding because the score couldn’t be presented as reliable evidence under any reasonable standard. The student was cleared, the department head was embarrassed, and the parents have a legitimate grievance they haven’t fully dropped.
Two things I’d say to any school considering using these tools formally:
1. Know exactly what you’ll do when a parent asks you to defend the methodology. If you can’t, you have no case.
2. The reputational damage from a false accusation is worse than the reputational damage from not catching an AI essay.
We now use detection only to prompt a conversation, never as evidence.
6 Replies
Join the discussion.
Log In to Reply
ok but the fact that a school used a Turnitin score to call a meeting before understanding what the score means is a failure of professional development, not just technology. we should not be deploying these tools without teachers understanding their limitations first. thats on admin, not just the tool.
25 years teaching. the burden of proof shifts in these situations in a way that's uncomfortable. once you make an accusation you're essentially asking the student to prove they didn't do something. with AI detection, that proof doesn't exist - neither side can definitively prove the origin of text. we need to be very careful about what we're starting when we make these calls.
filed three appeals this year involving detector scores. every single one went in the student's favour once we actually reviewed the process. at some point we have to ask what the tool is actually adding.
the detail that matters most in this story is that the student had documentation and the school still nearly acted on the score alone. process questions aside - what does it say about institutional culture that the burden of proof was entirely on the student? we should be building systems where the default is not assumption of guilt.
this is exactly why i was too uncertain to confront a student last year when a score came back high. something felt off about the essay but the score alone didn't feel like enough. reading this confirms I was right to hold back. the conversation-first approach makes sense.
Rebecca's point about the writing process is the whole answer. any integrity system without process evidence is guessing. the tool just makes the guessing feel official.