In Unit 1 of this module, “Creating and Aligning Program Vision,”  you began creating a blueprint with your collaborators for an online program by identifying the program’s “why” and end goal (program vision). In Unit 2, you analyzed whether it was feasible to begin the journey to program development. Now it’s time to develop the blueprint further as well as plan your journey to the final program destination. You can now create your Program Outcomes. create your plan for the structure of the program, and estimate how long it will take to finish development and launch. As noted throughout this Pressbook, you need to collaborate to be successful with program development, and this is especially true with respect to program planning. As you begin the early stages, include and communicate with stakeholders in your online program ecosystem regularly—it will support your ability to meet timelines and successfully launch the new program.

This unit focuses on the Policies and Procedures, Program Outcomes, Feedback, Continuous Improvement, and Sustainability elements of the Online Program Ecosystem. Read more about the ecosystem in Module 1, Unit 1: Collaborating to Create the Online Learner Life Cycle and its Ecosystem

Pre-Check: Assess Where You’re At in the Program Planning Process

Complete the branching scenario below to assess where you’re at in the program planning process.

What are Program Learning Outcomes?

Learning outcomes are statements that specify what learners will know or be able to do when they graduate (Ontario College Quality Assurance Service, 2019).

Writing Program Learning outcomes is about taking your program vision and expressing it so that it defines the destination for the students as they move through their learning journey during the program. Program Learning Outcomes inform decision-making about course design, so it may be useful to also consider course design at the same time given that course outcomes will need to eventually be mapped to program outcomes. Normally this is a linear process that begins with the program outcomes, which then inform subsequent curriculum development. It is this scaffolding that holds the academic elements of the program together. Watch the video below to hear one leader’s comments on important considerations for how Program Learning Outcomes guide program development.

Considerations for Writing Program Learning Outcomes

It’s worth noting that there is no difference between the process of creating program learning outcomes for online programs compared to in-person programs. There may be some differences in the outcomes themselves, for example, an online program’s outcomes can include the ability to understand and use certain technologies, but the process for writing the outcomes is the same. There are also important considerations to take into account depending on your institutional context. And, as always, taking a collaborative approach to writing your outcomes is best.

Colleges in Ontario, for example, differentiate between program standards (see Published College Program Standards, Ontario Ministry of Colleges and Universities), where the Ministry defines program outcomes as well as other required program elements such as essential employability skills, and program descriptions, where there are often developed program learning outcomes that serve as a “starting point for program development and review across the college system” (Ontario College Quality Assurance Service, 2021, p. 9). There is more flexibility–and work–involved with defining program learning outcomes for programs that fall under the program description category. Consult The Credential Validation Service (CVS) Handbook for New Program Submissions and Program Modifications (Ontario College Quality Assurance Service, 2021) prior to developing learning outcomes so that you know what is necessary for your specific program.

Accredited programs are another example where specific learning outcomes may be required. Consult the relevant accrediting body to determine the requirements prior to developing the outcomes.

In Ontario, there are also degree level standards to consider that, “identify the knowledge and skills expected of graduates of bachelor’s, honors bachelor’s, master’s and doctoral degree programs in Ontario” (PEQAB, 2021, p.24). Consult the Manual for Public Organizations (including Ontario Colleges) (PEQAB, 2021) to be sure the development team understands degree-level requirements. This will be important in later application to the Post-Secondary Education Quality Assessment Board (PEQAB) Board for approval.

While you will want to modify program learning outcomes to fit the vision for your online program, a quick environmental scan of websites for other program outcomes that are the same or similar will provide a start for thinking about your outcomes and this can save significant time.

Consider how the larger scaffolding of program outcomes will fit with the course-specific outcomes. For example, which course will introduce the program outcome?  Which course provides culminating assessments for the outcomes?  Keeping this in mind and note various connections that will facilitate the development of course outcomes and assessments later in the development process.

• • Subject matter experts who can provide expertise regarding the knowledge and skills necessary for the specific discipline or area of study
  • Subject matter experts who can support the use of an inclusive framework for developing learning outcomes so that they include a diverse perspective and a non-colonial approach.
  • Education consultants who have expertise in writing the outcomes and can support with language suggestions.
  • Potential students in the program

Writing Program Learning Outcomes

You will find that writing program learning outcomes can be challenging. Keep returning to your program vision to ensure your team has the destination in mind as the outcomes are being developed. And return to the resources provided by your institution and in this unit to support you in writing the outcomes.

Aligning with the appropriate domain of learning is also a challenge. Remember that outcomes can be presented and assessed at different levels multiple times throughout the program. Some outcomes may need to be reinforced. If your team does not have training in education, be sure to enlist experts in this area to provide advice. Finding the right words, specifically the action verb that describes what the students will learn, can be most challenging. Sometimes it feels as though all of the outcomes begin to sound the same after a while. Western University provides a simple guide for writing program-level learning outcomes that includes writing tips, taxonomies of verbs, and examples of program learning outcomes from the cognitive (knowledge-based), affective (values, attitudes, or emotion-based), or psychomotor (physical skills) based learning domains. A simple Google search for verbs for learning outcomes can help too, for example, you will find sites like Montana State’s Bloom’s Action Verbs for Learning Outcomes or SUNY Potsdam’s Action-Verb-List-For-Writing-Student-Outcomes.pdf (2003). Similarly, collaborating as a team with subject matter experts and education consultants will allow for multiple perspectives and ideas with respect to word choice.

It can also be a challenge to incorporate equity, diversity, and inclusion (EDI) in your program learning outcomes. New scholarship on this is emerging all the time, and there are resources available such as the notion of equity-focused principles, strategies, and resources at the University of Michigan Centre for Research on Learning & Teaching (Equity-focused Principles, Strategies & Resources). As a team, be sure to continuously reflect on the principles you want included with respect to EDI.

Remember, at the end of the writing process, the program collaborators and stakeholders should be able to agree that the Program Learning Outcomes capture the key knowledge, skills, and attitudes that a successful graduate of the program will acquire. As a final check, be sure to review if the learning outcomes align with the vision and description of the program. See this sample Program Learning Outcomes Retreat and the resource about Faculty-Driven Program Learning Outcomes to help with your strategy for the writing process.

Additional resources that support you in how to write effective outcomes can be found within an institution’s Centre for Teaching and Learning, or from accrediting and quality assurance bodies such as the Ontario College Quality Assurance Service (OCQAS) for colleges. We’ve included some of these resources below.

Write Program Learning Outcomes

Program Learning Outcomes and Quality Assurance/Sustainability

Module 4 addresses considerations for sustainability with specific reference to the unique needs of delivering online. At this stage, as you are planning the program, keep the various internal and external quality assurance processes in mind because you will need to provide evidence about how the program was created in a collaborative manner, how learning outcomes were developed and mapped, and how the online program is being evaluated through development and launch. In addition, formal review processes will be undertaken at various points in the program’s future, and this is included as part of policy, both institutional policy and Ministry policy. Listen as one online program development leader talks about their journey in considering how they looked to the future of the program from the outset of program design.

In the video above, Cebert touches on some key questions related to program outcomes, quality assurance, and sustainability: 

How will the quality of the program be formally reviewed in the future? 

There are formal processes such as degree renewal through PEQAB, or the College Quality Assurance Audit Process (CQAAP) that take place every five years. These audits have specific requirements related to learning outcomes and curriculum that must be included, such as breadth requirements or Essential Employability Skills, and they require provision of evidence that continuous improvement is being considered through various internal processes, including from the very start of the program delivery. Evidence is important—both in terms of process and continuous improvement. Keep all the materials, for example the meeting notes and the curriculum maps, that were created as part of the development process because they will be your evidence.

What should I consider for future quality assurance/sustainability from the beginning? 

Informal processes are necessary to ensure that the program vision and program learning outcomes continue to be achieved. Ideally, they include an annual process of reflection for the program team as well as a formal internal review process every 3 to 5 years. Considerations for both the annual reflection and internal review include, but are not limited to, questions such as:

• • Is the student voice included?  Whether through surveys or focus groups, student feedback is a part of any review process and serves as an important feedback loop for continuous improvement.
  • Are the relevant community partners or academic colleagues included?  These are the partners who will advise about changes in the environment, whether labour market or new scholarship that will inform changes to the curriculum for continuous improvement.
  • Are the roles and units that support your overall online program ecosystem involved? For example, if you have introduced new tools and technologies, have you consulted with IT or Educational Technology Units to see if your changes are sustainable?
  • Are equity, diversity, inclusion, and anti-colonial perspectives included in the feedback?  Having the right partners for providing this feedback is vital to ensure continuous improvement in this area.

Creating an Online Program Development Plan

You have your vision. You have your program description and your program learning outcomes, and you are collecting various pieces of evidence for future quality assurance processes. Now what?

The next step is to develop a plan for your online program’s development. Leaders at the institution will want to know when the program will launch for budget and recruiting considerations. The plan needs to include the timelines and milestones for development. For example, if you want to launch the new online program in an upcoming fall semester, and you know that internal and external approvals will take up to eight months and that you have to hire and train several new roles to develop the program, then you need to work backwards from your launch date to determine when various items will be needed.

The importance of timelines and setting expectations about timelines for approval is vital to planning effectively. Approvals will often take a year or more and are normally required before the development can take place. Creating a schedule for your institution of various approval milestones will support your planning, as this example illustrates:

Using the example above, for a new college diploma with an idea generated by February 19, 2021, the graphic below demonstrates the projected timeline.

Specific collaborators should be included in developing the plan as they can help you create realistic timelines for each part of the development process, particularly the development phase, which generally takes much longer than for in-person course development (anywhere from 4 months to a year for a course, not including any hiring or software/hardware integration that you may need to do). In this instance, they will include your subject matter experts and those roles noted earlier in Unit 2 of this module because they will be accountable for various deliverables and can advise on realistic timelines. If you have innovative or new-to-you aspects of your program, then review the information in Module 1, Unit 2, which will help you decide which partners you might collaborate with and the questions you can ask to help clarify what needs to be developed and the time and resources it will take. The Program Development and Implementation Workbook has a sample agenda for a planning meeting to determine what is available, what is then needed, and when things must be delivered to meet the launch date.

Consider the following when planning the online program development:

• Which collaborator will be required to complete which task?
• When do you plan to submit for Ministry approval?  For colleges, this also requires validation by the Credential Validation Service (CVS) and course descriptions are required and their development needs to be included in the plan. There will be other considerations like this that need to be included.
• What are the timelines for approvals and launch?

Unit Resources