Tested Walter AI Humanizer. It Works. That’s the Problem.
Before I get into this: I tested Walter AI because I wanted to understand what detection tools are actually up against. This is not a recommendation to use it for academic dishonesty.
Zero guidance from my district. Making it up as I go.
Tested Walter AI on 10 essays I had generated with ChatGPT (the raw output), then ran the Walter-humanized versions through GPTZero and Turnitin.
Results on humanized text:
– GPTZero: flagged 2 of 10 as high confidence AI. The other 8 came back low or uncertain.
– Turnitin: similar pattern. 3 of 10 high confidence, 7 low/unclear.
That’s detection accuracy dropping from 80%+ on raw AI text to 20-30% on humanized text.
Walter AI is good at what it does. Which means the detection tools we’re relying on are being actively defeated in the real world by students using exactly this kind of tool.
Policy implication: if a student uses AI then humanizes, our detection infrastructure catches roughly 1 in 4 cases. That’s what we’re working with. I think department heads need to know this.
5 Replies
Join the discussion.
Log In to Reply
admin: trust the technology. me: which technology are we trusting, the AI or the humanizer.
so Turnitin + GPTZero combined catch about 1 in 4 humanized AI essays. the ones that get caught are probably the students who didn't bother to humanize. we're running a detection system that primarily catches low-effort use and misses sophisticated use entirely. the inverse of what any rational policy would want.
the arms race this creates is: student uses AI → humanizes → submits. detection tools can't reliably catch it. so now we're banning humanizers too. which also can't be reliably detected. the technical solution path is a dead end.
the policy question this raises is whether humanizer tools should be in a different category than generative AI tools. a student using a humanizer on AI content is doing something meaningfully different from a student writing their own work and running it through Grammarly. but our current policies usually don't distinguish between them - or prohibit all AI use, making the distinction moot.
This is exactly what I've been trying to document. The sophisticated students - the ones who use AI then humanize - are invisible to detection. The lazy students who submit raw output get caught. We've built a system that penalizes effort applied to cheating inversely. If you work hard at cheating, you get away with it. This is not a small problem.