Educator explains Turnitin Similarity and AI Writing Scores
This instructional video, published on Apr. 2nd, 2024 by the LITE Center at Westcliff University, explains how to use Turnitin and interpret similarity and AI writing scores.
The following is a summary of Turnitin's similarity and AI writing scoring as explained in this video, omitting instructional content.
Similarity score: Comparison of student submissions to existing writing to detect repetition of words, phrases, or whole texts.
AI writing detection score: Calculation of the overall percentage of AI generated text
Similarity Score
How it works
The similarity score is based on checking text in a paper against a database of billions of web pages, student papers, documents and publications. This includes papers from your organization submitted to Turnitin previously. The score is a predictive indicator, where the higher the score, the more likely there has been academic misconduct (like plagiarism or contract cheating).
Investigation
Investigation
Screenshot from "Turnitin: AI and Similarity Reports" YouTube Video
For any paper with a similarity score, you can view the similar text identified as similar and compare against the matching source. If the matching source is not public, you can request access to the source paper.
AI Writing Score
How it works
Turnitin uses AI to detect text written by generative AI, like ChatGPT (3, 3.5, 4). It processes the document in segments (like sentences and paragraphs) and comes up with an overall average score based on the assessment of these segments. The video implies that it works by extracting the context from each segment, uses generative AI to rewrite the segment, and then compares the AI rewritten text to the original text to calculate the difference based on statistical patterns. Segments deemed likely to be written by AI count towards the AI writing score. The overall AI writing score is the percentage of text likely written by AI over all text processed for AI detection. Text that is not long form prose is omitted from AI detection and score calculation, such as poetry, bullet points, tables, and annotated bibliographies.
Investigation
When viewing a paper with an AI writing score, the text segment which is suspected of being AI written text is highlighted. Only the overall AI writing score is shown, with no further details per text segment available.
The narrator reiterates:
"Remember this report is a prediction not a definitive indication of AI generated text. However, the higher the prediction percentage the more likely your student did use AI to write this paper.
So just as with Turnitin's originality (similarity) detector does not definitively detect plagiarism, Turnitin's AI detector does not definitively detect AI generated text. Instead both alert you to areas of concerns you should review during your assessment process."