4.6 Assessing and revising your course

What happens after you have delivered your online course?

Creating a high-quality online course doesn’t stop with completing your design and offering the course to learners. Assessing the course itself after it has been offered, and making improvements in light of that assessment, is essential to crafting the best course experience for you and your learners. No matter how carefully you have designed things, running a course inevitably reveals aspects of your design and layout that can be improved, especially during its first offer.

Evaluating and revising your course is an easy step to neglect in the quality design process. However, we encourage you to take it just as seriously as the rest of your work on your course. Nothing is perfect the first time—iterative improvements to your online course will help you to create a durable high-quality learning experience as well as identify and develop new pedagogical and design strategies that you can use in future courses. Later in this module, we will help you think through the following key question pertaining evaluation and revision of your course.

Why assessing and revising matter

As an educator you are no doubt aware that every course is always a work in progress, and that opportunities for improvement, both big and small, present themselves in nearly every term a course is offered. Things are no different with online courses. However, as compared to in-class offerings, online courses may lend themselves to a more systematic approach to improvement. By committing to assessing and improving your course both within and between offers, you ensure the highest quality learning experiences for your learners going forward while also helping to increase your own satisfaction with your courses year over year.

A reflection from an Instructional Designer and Educational Developer at Queen’s University:

In my experience helping faculty members and other subject-matter experts develop de novo or redesigned online courses, it usually takes three to five iterations of the course for instructors to feel like their course is ‘perfect.’ In the first two iterations, be prepared for passionate learner feedback on what works and what doesn’t. After that, more detail-oriented learners tend to help point out areas that may need attention—either indirectly through their questions or in direct communication—in order to be genuinely helpful in improving the learning experience for others.

(Jenny Stodola, 2021)

Strategies for assessing and revising your course

Surveys and evaluations

The two most familiar and most important tools for assessing your online course are surveys and evaluations completed by participants. Chances are that your institution already requires your learners to complete overall course evaluations at the end of term, and you are probably already accustomed to gleaning information from these evaluations for course improvement. In many instances, however, official evaluations may not be specifically designed with online courses in mind. Therefore, while they often provide important feedback, it can be helpful to supplement them to get a more accurate picture of how well your online design and teaching strategies are working.

This is where surveys for your online course come in. Nearly all LMS providers allow you to create your own surveys and polls within the system to get additional feedback from your learners. You may wish to survey your learners near the beginning of the term, or at some point in the middle, to see how things are going with the course design, especially if you think there may be things you are able to change on the fly in response to their feedback.

Regardless of whether you use in-term surveys, however, creating at least one end-of-term survey for your learners is highly recommended. For your end-of-term survey consider the following tips to focus on online-specific issues and questions, especially those that won’t be covered in your official course evaluations.

• Ask about layout and information design in the course. Were things easy to find? Was the course intuitive to use?
• Ask about third party tools and other course technology if you are using them. Were there any technical problems or tool issues that learners ran into repeatedly? Can you use another tool or strategy, or improve your activity design to help reduce these issues going forward?
• Ask about discussions and interactions to get a sense of what learners found valuable and why. Could you use better prompts or design different ways of building social presence in your course?
• Ask about bottlenecks and problems that learners ran into with the content. Can some of your materials be made clearer? Can you spend more time during the next offer on certain difficult topics?
• Ask about the accessibility of the content, in terms of if they were able to access all the required materials for learning in the course in a format that met their academic accommodation needs. Prompt for details regarding specific challenges or gaps.
• Ask if the course provided a safe and accountable space for rigorous academic discussion. Prompt learners for positive experiences and situations that prevented them from full participation or where they may have felt uncomfortable. This can be a flag for further investigation.
• Ask learners if they felt represented in the material and to justify their response. An open-ended question could illuminate areas where learners might feel alienated by the way the content is presented, or opportunities to incorporate more explicit diversity in learning materials (e.g., in content, images, readings, videos, questions posed, assignment task, or outputs).
• Ask open-ended questions about what worked and didn’t work in the online environment. Give learners a chance to comment on anything that went well or poorly in the online-learning environment. Much of the most valuable feedback you will receive comes from open-ended question fields.

In addition to text-based surveys, you may wish to consider having individual meetings with learners to get their perspective on the course or even using a focus group to get a more complete picture of what might be improved in your next offering. Note, however, that a focus group should only occur after final grades have been submitted.

Examples of how evaluations and surveys could work

LMS metrics

Getting data directly from your LMS can be a helpful way to add objective information and analytics into the process of assessing your online course. Most LMS software today can give you a good picture of how learners are actually using your course. For example, you can usually find out

• which pages learners are visiting the most and for how long,
• where learners may be falling behind or falling off completely,
• how much learners are actually participating in forums and other interactions, and
• whether learners’ grades provide any information on difficulties they may be having with the course.

Every LMS is different in terms of the type and amount of data you can access. See the examples section below for some more specific information and a deeper dive into how you can use LMS data to help improve your course.

Examples of how LMS metrics could work