In this chapter

• Principles for assessment
• Asynchronous assessment options
• Synchronous assessment options
• Exams
• Academic integrity
• To go deeper

Principles for assessment

UBC describes principles for assessment in any setting:

• “Valid: Measures what it is intended to measure
  • The assessment should align with the intended learning outcomes (see Constructive Alignment for more information)
  • The assessment level should match the level of the learning outcome (see Bloom’s Taxonomy).  Consider carefully whether a multiple choice question can assess a high-level learning competency such as synthesis or judgement, and whether an essay or similar written question is needed to evaluate a low-level competency such as recall or comprehension
• Reliable: The results are consistent and dependable
  • Do different graders evaluate the student work the same way?
  • Are students likely to perform similarly if they retook the assessment in a few days?  Does the assessment depend on students happening to study minor details in the course or broad course themes?
• Practical: the cost, effort, and time frame are reasonable
  • The assessment needs to be workable for you, for TAs, and for students
• Authentic: it aligns with the target competency
  • e.g. can we assess teamwork using multiple choice quiz?”

The University of Calgary created a thoughtful explanation of key principles to consider for online assessments, including their reasons, how to enact the principle, and further reading. The main points are:

• Focus on learning (especially the most important aims of the course)
• Balance structure with flexibility (consider potential/known challenges students are facing during the pandemic)
• Provide clear instructions and quality, prompt feedback
• When possible, replace timed exams with other types of assessments
• Emphasize academic integrity (e.g., through conversations early and often, and an academic integrity statement in the syllabus); clarify expectations.

Asynchronous assessment options

Here are some options for structuring assessments over the course of the semester. We suggest relying less on marks from midterm and final exams, and providing more marks from other sources, such as:

• Brightspace quizzes; posted weekly on, say, Thursdays and due the following Wednesday
• Complete a problem set (not evaluated, just for learning purposes); questions & answers posted as PDFs
• Summarize weekly readings; submit as a Brightspace assignment
• Collaborative assignments; submit as a Brightspace assignment
• Discussion forums to summarize, analyze, engage, etc. Here is an excellent (and short!) summary by Gavan Watson, with references.

Synchronous assessment options

• Answer a series of questions in class – the instructor can review and expand on the answers or move on if the concept is well-understood. For example Kahoot and Menti are classroom response systems that can be used to gauge students’ understanding of material in near real time—they work as an online version of a clicker.
• Students can work in group to brainstorm, weigh evidence, propose solutions, etc.
• Submit one or two sentences identifying the main point of a concept on Menti.
• Please note: this section will soon be expanded. 🙂

Exams

In short

Houston, we have a problem. A new situation requires new approaches, and one of the most challenging parts of moving to remote teaching is figuring out how to evaluate students’ learning.

To make a long story short, if you wish to give a fairly traditional exam under these non-traditional circumstances, we recommend:

• a collaborative, open-book exam, or
• an individual, open-book exam, or
• if open-book exams cannot work, an exam using an online proctoring system

Whatever the exam style, administering the exam will require significant changes relative to typical, in-person exams. Most students live up to requirements around academic integrity, although an open book exam is probably going to be collaborative whether you want it to be or not, at least for a few students. Honest students may themselves feel cheated if they know that others have not upheld academic integrity standards and that is also a consideration.

Honour codes can also be used to emphasize and teach students about academic integrity and ethics; students can sign a declaration at the start of their exam attesting that they agree to follow the exam’s guidelines.

Practical details

To give a collaborative, open-book exam, set a small, maximum group size (e.g., 3). Students should not self-organize the groups. Communicate clearly with students about how answers are to be submitted (e.g., there is one submission per group, submit as a Brightspace “assignment”, answers will be checked using plagiarism software). You can require students to certify that they have participated equally in the preparation of answers.

A more detailed explanation

One of the biggest hurdles to offering a strong, remote learning course is understanding that exams cannot be administered in the traditional way. The practical barriers to assessing students effectively and fairly (and remotely) are significant for most professors. One approach is to consider alternatives to traditional exams; many options are explained in detail by the Taylor Institute for Teaching and Learning.

Recognize that there are no perfect ways to give a traditional exam under remote learning conditions. It’s harder still during a pandemic. One approach that has occurred to many is to give short, timed exams with the underlying premise that it is harder to cheat if time is very limited. We are skeptical that this is true. Moreover, administering that kind of exam in a remote learning environment is likely to create serious (and stressful) logistical challenges: all it takes is one legitimate technical hurdle and the whole, carefully timed process could fail.

The pandemic makes problems more challenging. We cannot have thousands of students in gyms writing exams for multiple courses in the morning, followed by a second cohort in the afternoon, and a third in the evening, and expect to maintain physical distancing requirements. Moreover, even if the university can enable such in-person exams for some courses, a change in the pandemic status (like the much-anticipated “second wave”) could derail plans with little warning. What would you then do instead to evaluate students?

Remotely proctored exams can now be administered at uOttawa through Respondus Lockdown Browser or Respondus Monitor. These remotely-administered formats require software systems that may be: highly invasive, unreliable or susceptible to academic fraud, too expensive, technologically difficult for students,  damaging to students’ experience in our courses, or all of the above. If you are planning to use this software, we strongly recommend following the recommended training, having a back-up proctoring method prepared (for students who do not consent to using Respondus), clearly describing the requirements in the syllabus (see template) and understanding the issues that arise from such tools (described below).

It’s a thorny problem.

A successfully administered midterm or final exam in a remote learning environment will:

1. Maintain academic integrity
2. Permit all students to participate equally, including from different time zones
3. Yield a reasonable measurement of student learning
4. Not impose unreasonable costs (monetary or otherwise) for students or the university

Academic integrity

Educators are understandably concerned about academic misconduct and minimizing academic misconduct involves more than putting in measures to stop it. Building a culture of academic integrity involves building trust through conversations and clear communication throughout a course, providing several opportunities for students to demonstrate what they have learned, making efforts to identify academic misconduct (e.g., plagiarized assignments, contract cheating), and institutional support that includes clear and robust policies, integrity campaigns with a positive focus, involving student leaders, and training programs for everyone in the university community.

Some of the essential aspects of academic integrity include:

• Having institutional policies that addresses teaching students about academic integrity and enforcing regulations
• Involving student leaders in conversations and policy-making
• Including explanations about academic integrity in course syllabi (available in the syllabus template in the Quick Start Chapter)
• Class conversations that inform, share expectations, and create an environment of academic integrity
• Requiring that students read and sign an academic integrity statement for the course overall and specific assessments
  • A template can be found in the Quick Start Chapter
• Including detailed instruction on assessments indicating what is and is not allowed; don’t assume students will interpret statements the same way you do. For example, saying that an exam is “open book” could mean that the students can consult:
  • Only the textbook
  • Any source, including the web
  • Classmates
  • Anyone, anywhere
• Designing assessments that mitigate cheating: authentic assessments (e.g., using real data on an assignment that can be used for another/real purpose), multi-stage assessments (individual written then oral), personal reflection, asking questions that require higher order thinking, which make cheating harder
• Deterring academic misconduct; please see options below.

Ways of deterring academic misconduct

1. Promoting academic integrity, as described above
   • Pros:
     • Builds a healthy classroom culture
     • Helps students who may not be aware of academic integrity/misconduct
   • Cons: some people will always cheat (the same is true in academia with plagiarism)
2. Spreading out the weighting of assessments, rather than having a few major ones
   • Pros: Addresses major reasons why students engage in academic misconduct
   • Cons: Like all methods, not a perfect solution. Having too many assessments can result in overload of work for students (workload estimator).
3. Using remote proctoring software
   • Pros:
     • May deter but does not stop academic misconduct
     • May help students feel less pressure to help their friends cheat
     • Zoom proctoring has been used as a method to proctor exams without using a formal proctoring software, particularly with an external webcam
   • Cons:
     • Many students feel this as an invasion of privacy (having to share their home environment)
     • Many students find that this type of video proctoring software increases text anxiety, which has had negative impact on performance
     • Creates equity issues with students who cannot purchase a webcam, don’t have wifi, or don’t have the latest software needed
     • Students can still find ways to cheat (e.g., using bathroom breaks, a secondary device, sticky notes on the computer)
     • Limits in question types that can be asked online; often still need pen/paper
     • Risks breaking the trust and community that we are working hard to develop with students—our future honours students, graduate students, colleagues, and co-professionals
     • Students with disabilities (e.g., physical movements) may get flagged more often than the average student, and as a consequence may feel that their privacy is more violated
     • Students with darker skin are frequently flagged more often because of the software’s difficulty seeing contrast.
     • When proctoring with zoom, students may be sending each other messages or using other software without having monitoring capabilities.
4. Addressing cases that are identified through institutional academic misconduct policies and processes
   • Teach teaching assistants what to do if they suspect cases of academic misconduct
   • Tell students how to report cases of academic misconduct (e.g., email to you, an anonymous form to fill out)

Options for exams that have been used to deter academic misconduct

1. Shorter time allowed for exams
   • Pros: in principle, harder to get answers in time
   • Cons:
     • Research shows that this method has little effect on cheating
     • Emphasizes a mode of rapid thinking rather than analytical thinking (either mode can be appropriate, depending on the context and purpose of the assessment)
     • Students can still have others answer for them or search for answers
     • Students are most likely to cheat on assessments that have a shorter turnaround time or account for a heavily weighted portion of the course grade
     • Does not account for technical issues that can arise
2.  Multiple versions of exams
   • Pros: harder to copy from someone else. Can make small differences that can help to identify cheating
   • Cons: difficult and time-consuming to make many equivalent versions
3. Not allowing backtracking (returning to a question once completed)
   • Pros:
     • Makes it harder for students to submit a question on a website that provides answers while they work on other questions
   • Cons:
     • Prevents students from working strategically on the exam as they often would (e.g., first answering questions they feel confident doing then leaving others to later)
4. Variable question sequences
   • Pros: Slightly hinders sharing answers
   • Cons: Has relatively little impact
5. Allowing students to take the exam only once
   • Pros: Analogous to an in-person exam
   • Cons:
     • Pressure to get the answers right on the first try is a factor in cheating;
     • They could do poorly if they have a bad day, are in a distracted home environment.
6. Delay score availability
   • Pros:
     • Indications of correct answers from the professor cannot be shared with others
   • Cons:
     • Removes an important learning mechanism
7. Delay or do not give feedback
   • Pros:
     • Correct answers from the professor cannot be shared with others
   • Cons:
     • Removes one of the most important learning mechanisms so substantially and negatively impacts learning and the majority of students in the course (honest students)
     • Can build a sense of distrust and harm the course environment

The following graph summarizes the perceptions of cheating on assessments, as reported by those who cheated.

Probability of cheating on assessments, as reported by those who cheated. Chart adapted from Dr. Martin Wielemaker. The data are from Bretag, T., Harper,R., Burton, M., Ellis, C., Newton, P., Van Haeringen, K., Saddiqui, S., and Rozenberg, P. ( 2019). Contract Cheating and Assessment Design: Exploring the Relationship, Assessment & Evaluation in Higher Education, 44(5): 676-691.

To go deeper

• University of Calgary’s Taylor Institute for Teaching and Learning, Five principles for meaningful online assessment
• University of Calgary’s Taylor Institute for Teaching and Learning, Academic Integrity and Online Learning
• University of British Columbia’s Chapman Learning Commons and Faculty of Science (contain suggestions for professors, communications with students, reporting mechanisms, etc.)
• Exams: Who are we leaving out? BCcampus
• A Guide for Academics – Open Book Exams
• 5 Tips for Using Take-Home Exams
• James Skidmore has shared extensive resources on assessment and academic integrity
• Create interactive videos using H5P through eCampusOntario’s H5P Studio or other methods, so that students can self-assess as they watch
• Weleschuk, Dyjur, & Kelly. (2019). Online assessment in higher education. Taylor Institute for Teaching and Learning Guide Series
• Wisniewski, Zierer, & Hattie. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers of Psychology. 10:3087. DOI: 10.3389/fpsyg.2019.03087
• Jopp & Cohen. (2020) Choose your own assessment – assessment choice for students in online higher education. Teaching in Higher Education, pages 1-18. DOI: 10.1080/13562517.2020.1742680
• Contract cheating and increasing academic integrity, Link
• Article: Fourteen Simple Strategies to Reduce Cheating on Online Examinations

Up next

The next chapter explains ways in which you can help students become proficient online learners, an environment that will be new for the majority.

Please feel free to contact us at any time with questions, suggestions, and concerns. In particular, we check this form each semester and will continue to update this guide as the situation evolves.