HOQAS review reports and student course feedback survey data
Student course feedback data
The student course feedback data was filtered by its correspondence to a HOQAS course review using the SCF-HOQAS concordance table developed. Each academic year (i.e. 2015/2016 and 2016/2017) corresponded to a reference period as the questions on the survey changed substantially between years. As well, though the response rates for these surveys were known to be low, the results were not filtered to only include high response rates as that data was unavailable at the time of analysis. 94 HOQAS review reports were linked to the 2015/2016 survey; 27 were linked to the 2016/2017 survey. Included reports were filtered to only include high certainty links (as indicated by a reliability scale value on the concordance table of ≥4).
| Survey reference period 2015/2016 | Survey reference period 2016/2017 | |
|---|---|---|
| HOQAS review reports counts (filtered) | 94 (73) | 26 (24) |
| Course sections (filtered) | 130 (104) | 30 (26) |
Means and standard deviations for each of the student course feedback survey questions used were calculated. For the 2015/2016 survey, items 6 to 16 (denoted as CORE 6 to CORE 16) were included; for the 2016/2017 survey, items 1 to 4 and 7 (denoted as CORE 1SF to 4SF and CORE 7SF) were included. As can be seen in the tables below, students rated course components fairly high, however, overall course ratings received mid-range ratings.
Table 1. Descriptive statistics for items 6 to 16 for the 2015-2016 academic year (n = 104).
| M | SD | |
|---|---|---|
| This course integrates educational technologies (e.g. online learning tools, e-classroom audio-visual equipment, etc.) in support of my learning. (CORE 6) | 4.18 | 0.60 |
| Course learning activities (e.g. lectures, discussions, practical work, group work, etc.) are varied. (CORE 7) | 4.06 | 0.62 |
| The course learning requirements for this course are clearly stated so that I know what to do to be successful in this course. (CORE 8) | 4.22 | 0.61 |
| The required course materials (e.g. textbooks, manuals, software, etc.) are used in the course. (CORE 9) | 4.04 | 0.68 |
| Opportunities exist to link the course material to the real world or workplace setting. (CORE 10) | 4.17 | 0.67 |
| The methods used to evaluate my performance are clearly outlined in writing. (CORE 11) | 4.18 | 0.61 |
| Course learning activities are linked to the course learning requirements. (CORE 12) | 4.25 | 0.52 |
| All of the course learning requirements are covered in the course. (CORE 13) | 4.23 | 0.54 |
| The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. (CORE 14) | 4.19 | 0.67 |
| The methods used to evaluate my performance are linked to the course learning requirements. (CORE 15) | 4.19 | 0.59 |
| Overall, please rate the quality of this course (CORE 16) | 3.66 | 0.96 |
Table 2. Descriptive statistics for satisfaction items 1 to 4 and 7 for the 2016-2017 academic year (n = 26).
| M | SD | |
|---|---|---|
| Overall, please rate the quality of this course (CORE 1SF) | 3.88 | 0.58 |
| Course learning activities (e.g. lectures, discussions, practical work, group work, etc.) are varied. (CORE 2SF) | 4.11 | 0.52 |
| Course learning activities are linked to the course learning requirements. (CORE 3SF) | 4.24 | 0.48 |
| All of the course learning requirements are covered in the course. (CORE 4SF) | 4.28 | 0.41 |
| Overall, please rate the effectiveness of your course professor (CORE 7SF) | 4.12 | 0.69 |
Correlations between student course feedback items and HOQAS review report scores
Given the ordinal nature of the scores from the HOQAS review reports (i.e. that they rarely occupied numbers between the minimum and maximum possible values), Spearman’s rho was calculated between all pairs of student course feedback items and HOQAS review standards. Spearman’s rho is a measure of the monotonic relationship between two variables; unlike Pearson’s r, which is a measure of the linear relationship between two variables, Spearman’s rho essentially determines the degree to which the values of two variables move in the same way (i.e. for positive values of rs, in the same direction; or, for negative values of rs, an inverse relationship/opposite directions).
For reference period 2015/2016, 11 SCF items × 43 standards were tested. 25 statistically significant (p < .05) relationships were found and the results are summarized in the table below:
| HOQAS standard | CORE | rs |
|---|---|---|
| 1.4 Course and/or institutional policies with which the student is expected to comply are clearly stated, or a link to current policies is provided. | 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | -.213* |
| 1.5 Minimum technology requirements are clearly stated and instructions for use provided. | 6. This course integrates educational technologies (e.g. online learning tools, e-classroom audio-visual equipment, etc) in support of my learning. | .246* |
| 7. Course learning activities (e.g lectures, discussions, practical work, group work, etc) are varied. | .215* | |
| 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | .220* | |
| 15. The methods used to evaluate my performance are linked to the course learning requirements. | .210* | |
| 1.7 Minimum technical skills expected of the student are clearly stated. | 7. Course learning activities (e.g lectures, discussions, practical work, group work, etc) are varied. | .231* |
| 8. The course learning requirements for this course are clearly stated so that I know what to do to be successful in this course. | .214* | |
| 12. Course learning activities are linked to the course learning requirements. | .217* | |
| 13. All of the course learning requirements are covered in the course. | .201* | |
| 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | .213* | |
| 15. The methods used to evaluate my performance are linked to the course learning requirements. | .230* | |
| 16. Overall, please rate the quality of this course | .243* | |
| 4.1 The instructional materials contribute to the achievement of the stated CLRs and weekly/unit learning outcomes. | 6. This course integrates educational technologies (e.g. online learning tools, e-classroom audio-visual equipment, etc) in support of my learning. | -.197* |
| 5.1 The learning activities promote the achievement of the stated CLRs, EKSs and weekly/unit learning outcomes. | 6. This course integrates educational technologies (e.g. online learning tools, e-classroom audio-visual equipment, etc) in support of my learning. | .222* |
| 7. Course learning activities (e.g lectures, discussions, practical work, group work, etc) are varied. | .237* | |
| 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | .201* | |
| 16. Overall, please rate the quality of this course | .221* | |
| 6.2 Course tools promote student engagement and active learning. | 9. The required course materials (e.g. textbooks, manuals, software, etc) are used in the course. | .200* |
| 13. All of the course learning requirements are covered in the course. | .234* | |
| 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | .230* | |
| 6.5 Links are provided to privacy policies for all external tools required in the course. | 6. This course integrates educational technologies (e.g. online learning tools, e-classroom audio-visual equipment, etc) in support of my learning. | -.265** |
| 8.1 Course navigation facilitates ease of use. | 15. The methods used to evaluate my performance are linked to the course learning requirements. | .218* |
| 8.4 The course design facilitates readability. | 10. Opportunities exist to link the course material to the real world or workplace setting. | .202* |
| 11. The methods used to evaluate my performance are clearly outlined in writing. | .206* | |
| 16. Overall, please rate the quality of this course. | .281** |
Note: * denotes p < .05; ** denotes p < .001
Grouping each sub-standard into its section, 11 SCF items × 8 standards sections were tested. 6 statistically significant (p < .05) results were found and the results are summarized into the table below:
| HOQAS section | CORE | rs |
|---|---|---|
| 3. Assessment and measurement | 7. Course learning activities (e.g. lectures, discussions, practical work, group work, etc.) are varied. | .203* |
| 11. The methods used to evaluate my performance are clearly outlined in writing. | .194* | |
| 14. The professor’s expectations for this course are clearly stated so that I know what to do to be successful in this course. | .269** | |
| 15. The methods used to evaluate my performance are linked to the course learning requirements. | .227* | |
| 8. Accessibility | 10. Opportunities exist to link the course material to the real world or workplace setting. | .218* |
| 16. Overall, please rate the quality of this course. | .203* |
Note: * denotes p < .05; ** denotes p < .001
For reference period 2016/2017, 5 SCF items × 43 HOQAS standards were tested. 6 statistically significant (p < .05) relationships were found and the results are summarized in the table below:
| HOQAS standard | CORE-SF | rs |
|---|---|---|
| 2.3 All CLRs, EKSs and weekly/unit learning outcomes are stated clearly and written from the students’ perspective. | 3. Course learning activites are linked to the course learning requirements. | -.450* |
| 4. All of the course learning requirements are covered in the course. | -.483* | |
| 4.4 The instructional materials are current. | 1. Overall, please rate the quality of this course | -.423* |
| 2. Course learning activities (e.g lectures, discussions, practical work, group work, etc) are varied. | -.424* | |
| 4. All of the course learning requirements are covered in the course. | -.424* | |
| 7. Overall, please rate the effectiveness of your course professor | -.414* |
Note: * denotes p < .05; ** denotes p < .001
Grouping each sub-standard into its section, 5 SCF items × 8 standards sections were tested. No statistically significant (p < .05) results were found.
