Assessment Design in the Age of AI

Creating assessments has changed dramatically since the release of ChatGPT. Our minds might first turn to the ways that learners may use AI, but these tools are also available to support instructors or learning experience designers in creating effective assessments. In this section, we are first going to talk about how AI may affect the decisions we make in designing effective assessments, and then we will consider all of these factors and how AI can be used to support the design and development of effective assessments.

For a comprehensive overview of many of these considerations, please take a look at this Generative AI and Assessment [opens in a new tab] resource page created by McMaster University’s MacPherson Institute.

Creating AI-Resistant Assessments

Assessments often serve multiple purposes in a learning environment, providing learners with the opportunity to practice, demonstrate, and reflect on their learning. Many educators have expressed concern about how AI use may undercut the process of learning, in pursuit of a final product. Below you will find a list of tasks that AI can more easily be used for and tasks that cannot be easily completed by AI (taken from this Generative AI and Assessment [opens in a new tab] resource by the MacPherson Institute) to help inform how we can re-design our assessments to limit the impact of AI.

What does this mean for educators?

When approaching the task of re-designing assessments to minimize unwanted impacts of AI use, there are shorter term and longer term options to pursue.

In the short term, educators can use in-class time and/or invigilation to ensure that learners are completing their assessments without the use of AI. It may also be helpful to adjust the requirements for the assessment or the grading rubrics used. Consider having assessments focus on the human processes involved:

• Learners can reflect on the metacognitive aspects of learning: How did the process of learning go? What challenges were faced? How were they overcome? Were there any surprises during the learning process?
• Learners can submit their incremental progress for feedback from the instructor or their peers. Each step of the assessment can build to the final deliverable, with grades emphasizing each step of this process.
• Grades can be assigned for connection to a learner’s own experience and to the course content covered in lectures, discussions, and readings.

In the long-term, educators may want to consider (re)designing authentic assessments. Based on the work of Grant Wiggins (1998, via Indiana University Bloomington), an authentic assessment:

• is realistic,
• requires judgment and innovation,
• asks the student to “do” the subject,
• replicates or simulates the contexts in which adults are “tested” in the workplace or in civic or personal life,
• assesses the student’s ability to efficiently and effectively use a repertoire of knowledge and skills to negotiate a complex task, and
• allows appropriate opportunities to rehearse, practice, consult resources, and get feedback on and refine performances and products

(Source: Generative AI and Assessment [opens in a new tab], MacPherson Institute, 2023)

To work through the process of designing authentic assessments, we highly recommend the workbooks created by the MacPherson Institute, which can be accessed in this Redesigning Your Assessment online resource [opens in a new tab].

Creating Assessments using AI

With some of the context generated using the activities listed above, you may discover that new, authentic assessments are better suited to your learning experience. Creating new assessments or redesigning old assessments to be more effective can be a time consuming and laborious process. In this section, we cover some ways that you as the instructor or learning experience designer may use generative AI to support your efforts.

If you’re new to creating prompts for generative AI tools, you may want to check out this Introduction to prompting generative AI like ChatGPT for teaching and learning [opens in a new tab] from Times Higher Education.

AI can be used to generate drafts of assessment instructions, rubrics, quizzes, examples for learners, and so much more. For example, the prompt template below can be used to generate a draft of an authentic assessment.

AI for Education [opens in a new tab] has prompt templates for numerous teaching-related tasks. We highly recommend browsing their library and using these prompts as a starting point for assessment design and creation.

Some tips for making these prompts your own:

• Ask ChatGPT or your tool of choice to align assessments with specific learning objectives from your course.
• Request multiple assessment types for you to choose between.
• Provide appropriate context for your learners, the goals of your learning experience, and specific examples you may be using.
• Provide as much detail as possible (e.g., a lesson plan or subsequent assessment that you’re building to)
• Request examples of ways that you can assess learners completing these tasks to give you a head start on creating rubrics.