4.1 Designing Assessments with Integrity

Try This!

Diagnose Your AI-Vulnerable Assessment

You are an assessment expert specializing in academic integrity and generative AI. Your task is to evaluate an assignment’s vulnerability to inappropriate or excessive use of AI, with a focus on whether it allows students to shortcut, rather than support, their learning. Begin by reviewing the full assignment instructions and any associated grading tools (e.g., rubrics, checklists). Based on these materials, complete a sample version of the assignment as if you were a student—this should reflect a typical, plausible use of generative AI. Keep the output realistic and aligned with expected student work.

After completing the sample, analyze the assignment’s AI vulnerabilities. That is, identify ways in which a student could use AI tools to generate a passing or high-quality submission while bypassing the deeper cognitive or developmental work intended by the task. Consider whether the learning outcomes assessed could be met with minimal human thinking or skill development due to AI assistance. Your goal is not only to flag vulnerabilities, but also to suggest why they exist and how they might be addressed.

Your analysis should incorporate the following guiding questions:

• Does the assignment assess a learning outcome where AI can support but not replace learning?
• Does the format of the assignment make AI use more likely (e.g., generic prompts, written formats, or predictable structures)?
• Are there ethical or equity concerns that might justify some AI use (e.g., accessibility, language support)?
• Do the instructions and evaluation criteria clearly explain how AI may or may not be used?
• Does the assignment include any mechanism for students to disclose or reflect on their use of AI tools?
• Can students reasonably be guided to use AI responsibly without undermining the assessment’s integrity?

Organize your response into three sections:

1. Sample Assignment Submission – A realistic AI-generated completion of the task.
2. Vulnerability Analysis – Specific opportunities for AI misuse or overuse and their implications for learning.
3. Recommendations – Suggestions to reduce vulnerability while preserving pedagogical value (e.g., revising prompts, adjusting outcomes, or scaffolding ethical AI use).

Brainstorm Ideas for Assessments (Power Prompt by Phillipa Hardman)

Use AI to create a range of knowledge checks, scenarios, and other assignments based on a learning outcome or goal. This is because LLMs are good at pattern recognition and repetition, which means they can quickly generate content variations using different question structures.

Create multiple suggestions on how to assess [learning outcome]. You must generate:

• 5 multiple choice questions, each question whose stem is a simple question with one correct answer and the remaining plausible distractors
• 5 scenario-based questions and activities, in which the scenario is [number] words long and the questions increase in difficulty
• 5 application-level questions and activities, in which ideas are applied to a hyper-specific context, organization, area of student interest, etc.

For each question, you must:

1. Provide sample answers and common errors or misconceptions.
2. State the related Bloom’s taxonomy domain of learning and level.
3. Explain the conditions in which it will be most effective, e.g., type of learner, mode of delivery, etc.

Design Formative Assessments for an Upcoming Assignment

Use Copilot to create prompts for formative feedback assessments for an upcoming assignment. See this sample prompt.

“You are an expert in creating prompts for formative feedback assessments in higher education using AI tools like Copilot. I want an interactive lesson focused on formulating effective prompts for generating formative feedback assessments that support student learning. Please give me key suggestions for improving my prompting technique when creating these assessments. After offering the suggestion, ask me a question to check my understanding. Wait for my response before proceeding. If my answer is incorrect, please correct me or provide further guidance with examples. Ensure that all questions are related to using formative feedback assessments to enhance teaching and learning in higher education. Do not give me more information until I confirm I understand or have answered correctly. As we go through the lesson, please provide me with examples of how well-formulated prompts can lead to effective feedback mechanisms and how these assessments can be used to monitor student progress and intervene when necessary. Conclude the session by summarizing what I have learned, highlighting my strengths and offering advice on continuing to practice and refine my prompting skills for creating impactful formative feedback assessments.”

Designing an AI Tenable Assessment

Create assessments using Copilot to provide insights into student learning. See the sample prompt.

“You are an expert in assessment design. Your task is to assist me with creating one or more assessments tailored to my students’ specific learning needs [insert course name]. Begin by identifying key concepts that need to be assessed, such as [insert core concepts like thesis development, evidence integration, analytical writing, etc.]. Based on the identified concepts, recommend a few specific types of AI-tenable assessments. This means that students should not be able to complete the assessment using only AI – they must add, adapt, or engage in a process of learning with AI, but also demonstrate their own unique human capabilities. Next, provide a clear indication of the appropriate level of use as well as the type or way that students would use AI. Next provide an appropriate way for students to share evidence of how they used AI. Next, recommendations on the optimal timing and methods for administering these assessments will be provided. For instance, suggest administering multiple-choice questions early in the unit to gauge initial understanding and using essay topics later as a summative check of students’ analytical skills as well as AI literacy skills. Finally, before creating a short list of the best AI-tenable assessments, ask whether these ideas are satisfactory or if more information is needed to refine the approach further.”

Try This! Exploring Copilot Prompts for Grading Tools

Use Copilot to explore various aspects of assessments. Follow the steps below for each activity. Insert search string information where required.

Create a Grading Tool 

Use Copilot to create a detailed, step-by-step rubric for an upcoming assignment.

• Step 1: Identify the assignment’s learning outcomes and key performance indicators. For a research paper, the learning outcomes might include critical thinking, research skills, and writing proficiency.
• Step 2: If possible, provide the assignment instructions.
• Step 3: Input these criteria into Copilot. Enter criteria such as “clarity of argument,” “use of sources,” and “grammar and style.”
• Step 4: Review and refine the generated rubric to ensure it meets your expectations. Adjust the weight of each criterion based on its importance to the overall assignment.

“Copilot, you are an expert in assessment of learning. Your task is to assist me with creating an analytic rubric using the ‘5+2 Level’ structure for my [insert course name] course. Begin by identifying the learning outcomes [LO1, LO2] being assessed and the specific criteria to be evaluated in the assessment {Criterion 1, 2, 3, 4]. I will provide you with assignment instructions. Next, ensure the rubric includes 5 distinct levels: Incomplete (Zero), Unacceptable, Developing, Acceptable (PASS), Emerging, Accomplished, and Exceptional. Next, set up the rubric structure. Assign total point values for each criterion, using multiples of 5, with a maximum of 20 points per criterion. Next, provide a detailed description for each criterion level. Ensure the description is specific, concrete, succinct, and aligned with the criterion and the level. Next, set point values using the provided charts, use the bolded number in each cell as the default point value, and consider including a range for greater differentiation within a level. Finally, format the analytic rubric using a table format, including all levels and grades on the x-axis and all criteria descriptors on the y-axis.”

Support Bias Reduction in Your Qualitative Student Feedback

Implement Copilot to analyze grading patterns and identify potential biases.

• Step 1: Collect a sample of feedback from graded assignments.
• Step 2: Use Copilot to analyze the feedback given. Review the Copilot-generated report to identify any patterns of bias or trends in comments.
• Step 3: Provide feedback on constructive, supportive comments with examples and invitational reflection questions.

“Please analyze the following sample of feedback I have provided to students on their graded assignments.”

<Insert your feedback samples here>

Please thoroughly review this feedback and identify any patterns of bias or recurring trends in my comments. Based on this analysis, provide detailed suggestions on how to make my feedback more constructive and supportive, including specific rewritten examples where applicable. Additionally, I will offer invitational reflection questions to help me critically assess and improve my feedback practices moving forward.”