Introduction

At the end of each term, students enrolled in courses within our department are invited to complete the University of Toronto’s course evaluation survey. In this chapter, we discuss the equity issues that can arise when student evaluations of teaching (SETs) are used inappropriately. To do so, we discuss potential sources of bias, how to make the best use of course evaluations, and how to use other forms of evidence (such as mid-course evaluations) as another tool for gathering formative feedback from students.

Awareness of Potential Sources of Bias and How to Make the Best Use of Course Evaluations

Student evaluations of teaching (SETs) create a challenging paradox when it comes to equity, diversity, and inclusion. On one hand, student evaluations of teaching, like any form of subjective assessment, can be influenced by a range of biases, including racism, sexism, and homophobia, among many other forms of prejudice (Heffernan, 2022). These biases raise serious concerns about the use of SETs as a form of assessment.

On the other hand, SETs are often the only form of voice provided to our students to shape their educational contexts, especially among our most vulnerable students, who may not have access to other forms of leadership to give voice and feedback. As such, efforts to downplay or eliminate the use of course evaluations as a method of feedback could serve to silence the voice of our students.

To ensure that faculty can use SETs as an important form of feedback on their teaching and learning, without fear that biases will negatively impact them or their careers, our department has made a commitment to treat student evaluations of teaching as just one of many sources of teaching effectiveness in our annual review process and when making tenure and promotion decisions.

• When evaluating the SET results of other faculty members, keep in mind the various factors that can influence evaluations (including course size, course topic, student motivation/circumstances, and prejudice/bias).
• Instructors are encouraged to reflect on and evaluate their teaching using a range of methods, including adopting a reflective and scholarly approach to teaching and learning, creating and regularly updating a teaching dossier, and engaging in multiple forms of teaching assessment (see https://teaching.utoronto.ca/resources/teaching-dossier/).
• Instructors who have concerns that prejudice or bias may be impacting the numerical ratings and/or comments on their student evaluations of teaching should start by reaching out to the Department Chair or Associate Chair, EDI, to express their concerns. The Chair can help the faculty member contextualize the comments within their portfolio materials, and as appropriate, within the context of the chair’s instructions to committee members and external reviewers.

Gathering Formative Feedback with Mid-Course Evaluations

The University’s Centre for Teaching Support & Innovation (CTSI) has developed an in-depth guide that provides Recommendations for Gathering Formative Feedback with Mid-Course Evaluations. The document discusses the benefits of conducting mid-course evaluations, introduces various models of assessment, and discusses strategies for their creation, implementation, and interpretation. Some key points from the document are highlighted below.

Common Models of Mid-Course Evaluations

One-Minute Papers

These unstructured free-response evaluations provide students with a short period of time to note any comments on the course thus far. Students might discuss their experiences with class assessments and activities, or provide feedback on the instructor’s teaching and explanation of concepts throughout the course.

Stop/Start/Continue

These evaluations prompt students to provide feedback on the course based on three categories: “stop”, “start”, and “continue”. “Stop” items describe ongoing course practices or activities that students feel are detracting from (or not contributing to) their learning. “Start” items describe things that students would like to see in the course. “Continue” items describe ongoing course practices or activities that students feel are beneficial to their learning.

Structured Evaluations with Course-Specific Questions

These evaluations include a brief number of questions prompting students to comment on specific aspects of the course, assessments, their learning experience throughout the course, or the instructor’s teaching.

Introducing Mid-Course Evaluations to Students

Students may be unfamiliar with mid-course evaluations, so outlining the process, expectations, and changes they might see is an important step in preparing them to provide meaningful and constructive feedback. When introducing these evaluations to students, the following points may be helpful to consider:

• Highlight how evaluations will be used, and what types of changes students might expect (or should not expect) to come from their feedback.
• Explain why you chose the questions you did, or why you chose a given format of assessment.
• Clarify how students’ anonymity will be protected and that their feedback and choice to participate will not impact their grades.
• Provide students with models of constructive responses illustrating both positive and negative feedback.
• Reinforce the idea that their feedback will be heard and seriously considered in making changes to the course.

Check Your Understanding

Resources

UofT Course Evaluation Interpretation Guidelines for Academic Administrators

A resource from the Centre for Teaching Support & Innovation (CTSI) highlighting core principles and effective strategies for interpreting course evaluation reports. The document specifically discusses the significance of course evaluation scores, student comments, response rates, and other contextual factors in the assessment of teaching.

UofT Guidelines on Developing & Assessing Teaching Dossiers

A resource from the Centre for Teaching Support & Innovation (CTSI) outlining guidelines and policies related to building, maintaining, and assessing teaching dossiers for faculty, administrative staff, and graduate students.

UofT Recommendations for Gathering Formative Feedback with Mid-Course Evaluations

A resource from the Centre for Teaching Support & Innovation (CTSI) introducing various models of mid-course evaluations and recommendations for their creation, implementation, and interpretation. This guide also discusses some benefits of conducting mid-course evaluations (beyond standard course-final evaluations).

References

Heffernan, T. (2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47(1), 144-154.

Contributions

Roles follow the CRediT Taxonomy. Names are listed in alphabetical order.

Conceptualization and Writing – Original draft: Kosha Bramesfeld

Writing – Review & editing: Amman Khurana

Supervision: Jessica Dere