Academic Quality: Comprehensive Program Review

Mohawk College: Centre for Teaching & Learning

Academic Quality: Comprehensive Program Review by Mohawk College: Centre for Teaching & Learning is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License, except where otherwise noted.

1

Welcome!

Source: The Global Exposure

This Comprehensive Program Review booklet has been designed with the aim to provide faculty and staff of each program area in the college with important information and resources regarding the review process.  This booklet serves as a guide to the college community involved in the program review process.  It is strongly suggested that you start with Glossary and Acronyms as you will frequently come across these acronyms throughout the course.

This open resource is intended to provide assistance and information to all stakeholders involved in the program review process.  It will also provide information for those who are planning a review of their program or who simply want to get information on how the process is conducted.

Each of the chapters examine various aspects of Mohawk College’s Comprehensive Program Review process and provides clear guidelines of the steps taken during the process. Through these chapters, you will also learn about the roles and responsibilities of the Program Review team and supports available to you.

Every effort has been taken to document the process as clearly, succinctly and accurately as possible. Program quality at the College supports continuous improvement of the process, and we therefore encourage feedback from you to enhance the information provided in this course. If you would like to provide feedback or need clarification about the content please contact the Program Quality Office.

Cebert Adamson

Dean, Continuing Education,

Academic Quality,

and Centre for Teaching & Learning

cebert.adamson@mohawkcollege.ca

905-575-1212 Ext. 4784

Mandira Raha

Program Review Coordinator

mandira.raha@mohawkcollege.ca

905-575-1212 Ext. 3358

Andreea Pohrib Baciu

Program Quality Specialist

andreea.pohrib@mohawkcollege.ca

905-575-1212 Ext. 2431

1

Acronyms ‘R’ Us

As with most disciplines and areas of specialization, Academic Quality has its own short forms for communicating. Although we use many of the same acronyms, or short forms, as other post secondary educators, we add a few that may be new to educators from other disciplines.

The flashcards allow you to refresh on common, post secondary acronyms as they mix in with Academic Quality-specific ones. Feel free to jump levels of difficulty and switch game mode to Match or try Gravity if you are already and acronym expert!

An interactive or media element has been excluded from this version of the text. You can view it online here: https://ecampusontario.pressbooks.pub/aqpr/?p=5

If you are unable to see the embedded Quizlet below, there is alternate access to the flashcards and all study modes.

Glossary

You may know all the acronyms, but there may still be some confusion on the roles and responsibilities of people and organizations behind the acronyms. We have provided a quick glossary of definitions, but please contact an Academic Quality team member for further clarification.

Associate Dean

College Educator Development Program

Course Ouline Mapping and Managment System

College Quality Assurance Audit Process

Centre for Teaching & Learning

Credential Validation Service

Essential Employability Skills

Institutional Research

Ontario College Quality Assurance Service

Program Advisory Committee

Strategic Mandate Agreement

2

Context for Quality Assurance: The Ontario Perspective

Learning Outcomes

Examine program quality within the Ontario College System

- Outline Mohawk College policy

- Describe the relationship between the Ministry and Ontario Colleges with reference to program quality

- Define agencies involved with Colleges’ program quality

“Productivity and efficiency can be achieved only step by step with sustained hard work, relentless attention to details and insistence on the highest standards of quality and performance.” ~ J.R.D Tata

Program Quality at Mohawk College

Program Review process at Mohawk College integrates elements of the College Quality Assurance Audit Process (CQAAP) standards. The review process at the College ensures that all issues of quality are addressed on a regular basis. In order to achieve this, several policies have been developed and committees established or revised to meet these standards. These policies reside on the Mohawk College website and are accessible to the public.

The Program process designed to advance strategic plan and program quality. Mohawk College Program Quality Policy (AS-2000-2013) establishes college framework for the program review process with focus on continuous quality improvement.

The strategic priorities such as Student Success, Graduate Success, Collaboration and Partnership, Community Leadership, and Strong Foundations are assessed in program review, and outcomes to support each priority are developed based on feedback from each stakeholder group.

Purpose of Program Review

Hold for quote from feedback survey.

Program reviews yield numerous benefits and provide an opportunity to evaluate, analyze, and assess the content, currency, direction, and quality of a program at Mohawk College. The process fosters academic excellence, helps programs to determine how to raise the quality of its offerings, and provides guidance for curricular and administrative decisions in support of continuous program development.

In its strictest sense, program review brings about improvement through the collection of evidence about the quality and effectiveness of programs. Feedback is sought through shared reflections and mutual dialogue about the programs’ current quality and future direction, and through constructive feedback involving all stakeholders.

A variety of college internal and external stakeholder such as students, the program area team, program advisory committees, and industry partners feedback are solicited during the program review process at the college.

Provincial Background

The Ontario government introduced The Colleges of Applied Arts and Technology Act, in 2002. This Act required all Ontario public colleges to implement two elements:

1. Quality assurance at the individual college level, and

2. A self-regulating process at the system level.

Subsequent to this legislation Ontario College Quality Assurance Service (OCQAS) , the accrediting agency was created to maintain quality assurance processes in Ontario colleges in a self-regulatory environment.

OCQAS is an intermediary between Ministry of Training, Colleges and Universities (MTCU) and Ontario colleges. OCQAS is responsible for:

1. Ensuring quality at the program level through the Credential Validation Service (CVS), as well as

2. At the institutional level through the College Quality Assurance Audit Process (CQAAP)

It is OCQAS responsibility to oversee consistency of quality across Colleges.

Since 2005 Ontario public colleges with OCQAS, successfully developed a strong, well received, and effective quality assurance system. Although the existing audit model is effective, Ontario colleges continue to experience increasing pressure of the changes in the higher education environment globally. Need for new practices of instructional delivery, improved accountability, and globalization of higher education are only some of the factors that are increasingly influencing the higher education environment in Ontario colleges. To broaden global outreach and to meet the increasing demand for accountability Ontario public colleges decided to explore the adoption of an accreditation model (Ontario College Quality Assurance Service, July 2015).

To read more about the move to Accreditation click here.

Bringing It Back to You

The provincial and institutional perspectives are important, but Program Review has personal | professional benefits for each individual educator. Capturing some of these benefits will help others persevere through the process and understand the value. Add them here:

An interactive or media element has been excluded from this version of the text. You can view it online here:
https://ecampusontario.pressbooks.pub/aqpr/?p=45

Introduction to Program Review

In 2002, MTCU provided Ontario College’s greater autonomy with program development, curriculum enhancements and quality assurance through the implementation of the Minister’s Binding Directive for the Framework for Programs of Instructions.

This Framework, explains both Ministry and College responsibilities for maintaining program quality at all credential levels. It includes the credentials framework for Ontario College Certificates (OCC), Ontario College Diplomas (OCD), Ontario College Advanced Diplomas (OCAD), and Ontario College Graduate Certificates (OCGC).

Ministry Requirements

The directive within the Framework for Programs of Instruction outlines specific criteria and processes including a review of all programs of instruction at every Ontario college. The directive indicates that:

“Colleges are to establish mechanisms for the review of their programs of instruction to ensure ongoing quality, relevancy, and currency. A college’s policy on quality assurance for programs of instruction is to be publicly available”.

Credential Validation Service (CVS)

The mandate of the Credential Validation Service (CVS) includes:

- providing reasonable assurance that all programs of instruction, regardless of funding source, conform to the established Credentials Framework and are consistent with accepted college system nomenclature and/or program titling principles; and,

- maintaining the integrity of the credentials and protecting the interests of students and employers who require a reasonable guarantee of consistency and quality in Ontario’s programs of instruction.

CVS approves the program vocational learning outcomes (VLOs) for all proposed programs of instruction. They are also involved in the updating of program standards.

College Quality Assurance Audit Process (CQAAP)

College Quality Assurance Audit Process (CQAAP) is an institutional level process that involves the regular and cyclical review of each college’s quality assurance measures. Program Review is one such measure in place in each college.

Colleges are responsible to maintain the quality of their programs and are audited every five (5) years.

All Ontario colleges are audited based on six (6) standards.

An interactive or media element has been excluded from this version of the text. You can view it online here:
https://ecampusontario.pressbooks.pub/aqpr/?p=45

Read more about the above 6 Standards.

Show What You Know

An interactive or media element has been excluded from this version of the text. You can view it online here:
https://ecampusontario.pressbooks.pub/aqpr/?p=45

Key Takeaways

To be developed collaboratively with PQ peeps

- First

- Second

3

Quality Standards

Learning Outcomes

Examine program quality within the Ontario College System.

- Describe Mohawk College’s quality assurance strategy.

- Define the four main categories of Quality Standards.

Quality Assurance – The Strategy

At Mohawk College, the quality assurance strategy includes a combination of the standards, policies, people, and processes which ensures quality in every area of the college. Program review is a component of the larger college quality assurance strategy.

Quality Standards

The program quality standards are reinforced by the College’s Vision, Mission, Strategic Directions, Academic Plan, and the standards and guidelines established by the MTCU and other accreditation and professional bodies. These quality standards have been developed to be applied to all programs. These standards fall into four main categories which Curriculum Development Specialists will use to form the foundation for the program review process:

Program Learning Outcomes

Learning outcomes represent culminating demonstrations of learning and achievement. In addition, learning outcomes are interrelated and cannot be viewed in isolation of one another. As such, they should be viewed as a comprehensive whole. They describe performances that demonstrate that significant integrated learning by graduates of the program has been achieved and verified.

The Curriculum

The curriculum design refers to the arrangement of the elements of a program into a substantive entity. That is, it refers to the naming of the arrangements of certain components, including its title, mission, aims, the courses that make up the program and the explanation of those course outlines, generally in fully developed course outlines. The design often is influenced by the curricular approach and the philosophical orientation of the policy makers and those responsible for the design of the curriculum.

Delivery Methodology

The success of a program will be derived largely from the methods of instructional delivery employed. It is therefore important that program development and/or reviews, consider the appropriate mix of leading and cutting edge delivery options that will allow learners to develop their skills, knowledge, and experiences.

Assessment Methodology

Assessment is about grading and reporting student achievements and about supporting students in their learning, hence should be carefully considered during the program planning stages. Assessment should also be relevant to learning outcomes and consequently be current.

It is expected that the college will be familiar with, and follow, current, accepted, evidence-based practices and research related to the quality of programs and student learning, and have in place related policies and practices (Program Curriculum Policy) that are consistent with this evidence and research.

4

Program Review at Mohawk College

Learning Outcomes

Apply the program review expectations to their specific program area.

- Explain the purpose and goals of program review.

- Explain elements of both Comprehensive and Annual program reviews.

Goals of Program Review

Academic Program Review is one cornerstone of Mohawk College’s quality assurance strategy.

This is conducted primarily to:

- Assess the program against established provincial and college standards.

- Examine innovation or alternative teaching/learning/assessment practices.

- Examine issues related to student access, success and satisfaction.

- Ensure that all program/course changes are informed by student and industry feedback.

- Determine if the program adequately prepares students for changing job markets, market demands and/or for transitions to further study.

- Establish and implement an action plan for the program.

Types of Program Review

There are two levels of program reviews conducted at Mohawk College:

- The Comprehensive Program Review (conducted every five years), and

- The Annual Review

Note: Programs that undergo a Comprehensive Review are not required to complete an Annual Review in the same year.

Comprehensive Program Review

An extensive Comprehensive Program Review conducted every five years evaluates the program against the program quality standards and industry trends. This provides program area faculty the opportunity to reflect on the accomplishments, challenges, and overall effectiveness of the program they represent, exception being undergoing external accreditation.  The College ensures that programs undergoing external accreditation provide robust data and information that aligns with the College’s program quality requirement, and where such requirements are inadequate, Program Quality Team (PQT) works with program areas to ensure program quality is met.

Comprehensive Program Review is a five stage process and is estimated to be covered over a twelve (12) month period, involving several internal and external stakeholders.

Figure 1 Briefly outlines the stages and some of the activities included at each stage. If you prefer plain text, it is available.

Figure 1: Comprehensive Review Stages

Resources for Conducting the Program Review

The college makes available numerous resources to the Program Review process through various units. The Curriculum Design Specialists (CDS) and Program Quality Team (PQT) will support you to identify important stakeholders to involve in the review process, including but not limited to:

- Program Quality Team

- Centre for Teaching & Learning

- Institutional Research

- Finance

- Pathways and Credit Transfer

- Marketing and Recruitment

- Student Success Advisors

- Accessible Learning Services

- Indigenous Services

- International and Partnership

The Comprehensive Program Review Process

Figure 2 below illustrates the timelines, operational stages, and process that are implemented during the comprehensive program review process.

An interactive or media element has been excluded from this version of the text. You can view it online here: https://ecampusontario.pressbooks.pub/aqpr/?p=65

1

This is where you can add appendices or other back matter.

2

Plain Text: Stages of Comprehensive Review

Pre-Planning & Orientation

- PQT and AD establishes PRT

- PQT Coordinates orientation session for ADs and Program Review Leads (PRLs)

- CDS meet with PRLs to plan stakeholder sessions & review process

Data Collection

- CDS and PQT review course outline and reports from COMMS

- CPC data is collected by PWT

- IR provides environmental scan data (Academic & non-academic)

- PRT schedules and holds student and employer | industry stakeholder consultations to gather data

- Other pertinent data is collected

Analysis

- PQT & CDS host Analysis workshops: PLOs, Curriculum, Delivery, and Assessment

- Analysis of all data collected to inform recommendations and action plan

Report

- PRL with Report Writer prepare prelininary report

- Report reviewed by relevant stakeholders

- Submission to:

- AD

- Dean

- PAC

- PQT

- Office of VPA

- VPA meets with PRT to discuss Final Report

External Verification

- External team validates review report