All Content

Educator explains Turnitin Similarity and AI Writing Scores

 This instructional video, published on Apr. 2nd, 2024 by the LITE Center at Westcliff University, explains how to use Turnitin and interpret similarity and AI writing scores.




The following is a summary of Turnitin's similarity and AI writing scoring as explained in this video, omitting instructional content. 

Similarity score: Comparison of student submissions to existing writing to detect repetition of words, phrases, or whole texts.

AI writing detection score: Calculation of the overall percentage of AI generated text

Similarity Score

How it works

The similarity score is based on checking text in a paper against a database of billions of web pages, student papers, documents and publications. This includes papers from your organization submitted to Turnitin previously. The score is a predictive indicator, where the higher the score, the more likely there has been academic misconduct (like plagiarism or contract cheating). 

Investigation

Screenshot from "Turnitin: AI and Similarity Reports" YouTube Video



For any paper with a similarity score, you can view the similar text identified as similar and compare against the matching source. If the matching source is not public, you can request access to the source paper.

Its expected that there be some matches even if the student has used quotes and references correctly.  Commonly used names and phrases contribute to the similarity score. For example "according to the Environmental Protection Agency's website, https:/www.epa.gov" or "Journal of the Air & Waste Management Association", or references like "(Bel Hadj Ali et al., 2020)".

AI Writing Score

How it works

Turnitin uses AI to detect text written by generative AI, like ChatGPT (3, 3.5, 4). It processes the document in segments (like sentences and paragraphs) and comes up with an overall average score based on the assessment of these segments. The video implies that it works by extracting the context from each segment, uses generative AI to rewrite the segment, and then compares the AI rewritten text to the original text to calculate the difference based on statistical patterns. Segments deemed likely to be written by AI count towards the AI writing score. The overall AI writing score is the percentage of text likely written by AI over all text processed for AI detection. Text that is not long form prose is omitted from AI detection and score calculation, such as poetry, bullet points, tables, and annotated bibliographies.


Investigation

Screenshot from "Turnitin: AI and Similarity Reports" YouTube Video



When viewing a paper with an AI writing score, the text segment which is suspected of being AI written text is highlighted. Only the overall AI writing score is shown, with no further details per text segment available. 

The narrator reiterates:

"Remember this report is a prediction not a definitive indication of AI generated text. However, the higher the prediction percentage the more likely your student did use AI to write this paper. 

So just as with Turnitin's originality (similarity) detector does not definitively detect plagiarism, Turnitin's AI detector does not definitively detect AI generated text. Instead both alert you to areas of concerns you should review during your assessment process."