4.3 Grading in the AI Era

Grading student assessments is a crucial part of teaching and learning. Grading with integrity means evaluating student work honestly, fairly, and consistently. It involves maintaining confidentiality, applying avoiding bias, and offering constructive feedback. Upholding academic standards is crucial to fostering a trustworthy and respectful academic environment.

Avoiding AI for Grading Student Work

There are many problems with using generative AI to grade or provide feedback on student work (Ardito, 2023):

• AI can be unreliable: AI may not tell the difference between important and unimportant context or even fact from fiction.
• AI can’t explain its decisions: Without real-world context beyond the data it has been trained on, it cannot evaluate the quality of an argument as a human marker could
• AI systems can be biased: The AI system could treat some groups of people favourably based on their characteristics. This can result, for example, in a bias against non-English speakers.

Generative AI does not know your course, students, or learning contexts. It will not provide grades or feedback tailored to the complex contexts of your course. In addition, the tailored and supportive feedback you provide invites students into a personalized conversation about their learning.

Students also have the right to be informed about how their work is being copied and re-used, including generative AI tools.

For these reasons, Conestoga’s AI Do’s and Don’ts document advises faculty: Faculty should not use AI to grade student work.

AI Detection in Grading

AI detection software uses AI to help identify potential misuse of AI in academic work. They use a combination of large data sets, natural language processing, statistical analysis, and pattern recognition to detect anomalies typical of AI-generated text (NIST, 2024). For some types of AI-generated content, metadata can include information about generative AI content, and provenance data tracking may also help assess content’s authenticity and integrity (NIST, 2024).

However, these detection and tracking tools have limitations and should be part of a broader strategy to maintain academic integrity. Detection tools are not foolproof and can produce false positives. They may favour some students over others (Khalid & Daumé, 2023). They may also reinforce a culture of policing rather than learning. As more AI-generated writing becomes part of the training data, it may be harder to source authentic human-written datasets. As language evolves, anachronisms may also be labelled as AI-generated (Artido, 2023).

When distinguishing between human and AI-generated work becomes increasingly complex, a “cat-and-mouse” game, some authors argue for an alternative to AI-grading tools. As Artido (2023) states, “[I]t is imperative to design educational systems and assessment methods that do not depend on the flawed premise of being able to distinguish AI-generated content from human work accurately. This shift is not just a practical necessity but a strategic adaptation to the rapidly evolving landscape of AI capabilities, ensuring that educational integrity is upheld through more robust and future-proof methods” (p.4).

Faculty should recognize that AI detection software can be useful but not infallible.

• Learn about Turnitin’s AI detection checker: Familiarize yourself with Turnitin’s AI checker and its detection capabilities to understand better how it identifies AI-generated content.
• Supplement with other methods: Use detection software as one of several approaches to ensure academic integrity. Combine it with other methods like using authentic assessments and meeting with students when academic misconduct is suspected.
• Educate on software limitations: Inform students about the limitations of detection software and the importance of personal integrity.
• Provide resources: Offer academic integrity resources such as writing centers, tutoring, and counselling to support students academically and personally.

Turnitin’s AI Checker

Teaching and Learning provides guidelines on Using Turnitin’s Artificial Intelligence (AI) Detection Tool and the Process Guide for Navigating Potential Academic Offenses.

They also provide three scenarios in which you may need to contact students to express concerns about suspected non-permissible use of AI. See Email Templates Responding to Suspected Student Use of Generative AI.

Next page >>