Scores, feedback, communication, and debriefing of faculty
When the HOQAS review of a course was completed, a report would be sent to both the faculty member responsible for the course and the faculty member’s chair. Given the stringent criteria for a course being considered as having met the HOQAS/QM standards—a score of at least 85 out of 99 and a full score in each of the 21 standards considered as essential—most courses did not initially succeed. The HOQAS review process tracking spreadsheet contained final scores for 187 reviews (M = 57.2, SD = 15.9). The maximum score was 99, while the minimum score was 21. Only 11 of 187 reviews (6%) achieved a score sufficient to be considered standards compliant. The narrower set of 140 reviews (i.e. not including the audits of revisions) that were parsed from the HOQAS review reports contained only 4 courses with a score sufficient to be considered standards compliant and no courses that met all of the essential standards.
Many of the faculty interviewed were not familiar with the HOQAS review process nor the HOQAS/QM standards prior to submitting courses for review. As such, the low scores and determinations of being non-compliant with the HOQAS standards took faculty members by surprise. Part of the challenge of the scoring and the standards is that faculty reported that they were either initially not available or not well explained; thus, even if they had wanted to faculty would have had a hard time designing their courses to be standards compliant. The challenges that faculty were having with knowledge and understanding of the standards were addressed somewhat with the introduction, towards the end of the first year of the program, of a learning management system template that aimed to provide an annotated structure—and in some cases, standardized content for some of the standards sections—to support faculty in meeting the HOQAS standards. Many faculty reported such a template as being helpful in general, though specific feedback on it included the redundancy of some sections and a potentially overly complex navigation structure. Faculty reported appreciating having a starting point for the design and development of the online components of their course, with some saying that they are still using the template, or their own variations of it, despite the HOQAS review program having been put on hiatus.
Faculty reported having submitted their courses in the hopes of receiving feedback on how to improve their online components; with the College’s continuing conversion of face-to-face courses to hybrid or online, they were looking for guidance on how to best use their online time. Instead, they received scores and reports which were described by many as unhelpful. One faculty member described his low score as demoralizing, as he had thought that the course he submitted was of good quality; others reported faculty worry about potential repercussions from receiving a low score, since the department chairs were also included in correspondence about the results of the HOQAS reviews. As well, while some faculty were hoping for a collaborative feedback process, they reported feeling as though the HOQAS review process had little communication, noting periods of silence between submission and receiving their—usually low scoring—HOQAS review report, resulting in a loss of motivation for participating further in the process, including doing revisions for their courses. Institutional leadership recognized that this was the result of a focus on completing reviews; with the goals that had been set, there was no time for a more collaborative, and therefore time-consuming, process.
