How Does Turnitin Detect AI Writing? The Technical Breakdown

I’ve been reading up on Turnitin’s technical documentation and wanted to break down how their AI detection actually works, because understanding the method helps us interpret the results better.

Turnitin’s AI detection model is trained on a large dataset of both human-written and AI-generated text. It learned the statistical patterns that distinguish the two. When you submit a paper, the model analyzes the text at the sentence level and assigns a probability score to each sentence.

The overall score you see is an aggregate of all those sentence-level predictions. This is why you sometimes see papers flagged at 40% or 60%. It doesn’t mean the whole paper was AI-generated. It means that proportion of sentences matched AI patterns according to the model.

The model primarily looks at what researchers call “distributional properties” of the text. AI language models generate text by predicting the most likely next word based on the preceding context. This creates patterns in word choice, sentence structure, and topic flow that are statistically distinguishable from human writing.

One important thing Turnitin has disclosed: their model has a false positive rate of less than 1% when the AI probability is above 80%. But at lower thresholds like 20-50%, the false positive rate increases significantly. This means you should have high confidence when the score is very high, but lower scores require more investigation.

Turnitin also notes that their model was trained on specific AI systems (primarily GPT-3.5 and GPT-4). As newer models emerge and AI writing becomes more sophisticated, the detection model needs continuous updating.

One limitation: the model doesn’t work well on very short texts (under 300 words) or on text in languages other than English. If you teach in a bilingual Canadian school, keep this in mind.

What questions do you have about how AI detection technology works?