Key Findings: Open Badges

Open Badge Overview

“Open Badges are visual tokens of achievement, affiliation, authorization, or other trust relationship sharable across the web” (Discover Open Badges, 2018). These digital representations contain credentialling information, or metadata, based on an open technical standardized format created by the Mozilla Foundation in 2011 and currently managed by IMS Global. This metadata is determined by the badge creators and can include a badge name, links to a description and evidence, date of issue, criteria, issuer, issue date, expiration date, recipient, tags, alignment, and third-party endorsement. Open Badges can be easily shared in social and online networks and curated by the earner in a collection or portfolio for career, personal, and/or professional advancement. Open Badges represent one solution for individuals to aggregate learning achievements and associated evidence of learning from multiple sources in portable, digital, interoperable, and verifiable ways (Badge Alliance Endorsement Working Group, 2014).

Because Open Badges are relatively new, the research on their use within higher education is limited. However, case studies do exist that indicate one of the biggest obstacles to implementing badges as a form of recognition is the lack of perceived value by institutions and employers (Hickey, Willis, and Quick, 2015). However, a University Professional and Continuing Education Association (UPCEA) report based on data from 190 post-secondary institutions found that alternative recognition is rising in both value and popularity in non-traditional offerings and with non-traditional learners who need to share outcomes across their professional networks (Fong, Jangzow, and Peck, 2016). This finding is reflected in our badging evaluation reports as well, with badges having a greater value and uptake in non-credit faculty development and continuing education course implementation.

The eCampusOntario Educational Technology case studies involved badging explorations in professional development, faculty development, continuing education, credit and non-credit offerings, undergraduate graduate programs, and co-curricular experiences. Of these, more badges were issued by those institutions targeting continuing education, faculty development, and co-curricular experiences. For many other institution partners, the limited pilot period did not allow time to issue a large number of badges.

Summary of Badges Created, Issued, and Accepted

Participating institutions created badges that best represented their projects. Once potential earners submitted their evidence and a reviewer signed off on the demonstration of skills to fulfill the badge’s requirements, CanCred issued the badge through an email. Earners accepted the badge by clicking on a link and were then able to add it to their “backpack,” post it on LinkedIn, or port it to other comparable systems. As administrators, eCampusOntario was able to track the process across its partners projects. Tables 1.1, 1.2, and 1.3 show the total badges created, issued, and accepted during the Sandbox Evaluation period.

Table 1.1: Badges Created, Issued, and Accepted during Sandbox Evaluation Period

Table 1.2: Badges Created, Issued and Accepted by Post-Secondary Education Partners Partners during Sandbox Evaluation Period

Table 1.3: Badges Created, Issued, Accepted by eCampusOntario during Sandbox Evaluation Period.

Table 1.1 shows an overall badge acceptance rate of 41%. Two of the highest acceptance rates were from Durham College and Georgian College. Both those partners indicated that they created a robust orientation to what badges are, which may have contributed to this success. In particular, Georgian College’s badge implementation was geared toward its faculty development program, with badges being awarded not only for this program, but also described as potential recognition for faculty as learners. Georgian College notes in its report that the badges were issued to colleagues who already have an established relationship with the Teaching and Learning Centre, which likely explains the higher-than-average acceptance rate. Durham College issued its badges within non-credit continuing education programming; anecdotal feedback was that learners were anxious to receive recognition of their badges to share on social media and LinkedIn.

The Value of Badges for Earners, Educators, and Employers

The overall level of uptake on badges issued indicates that earners are not fully aware of what these alternative forms of recognition represent. Survey results from one of our partners, Ryerson University, verified this theory, with a large number of badge earners (whether they accepted the badge or not) reporting that they were unclear about what a badge was. This feedback indicates the need for strong orientation to the value and the portability of Open Badges. A great tip by one partner institution was to include a linked resource on what a badge is and what it can represent as part of the badge-issuing message.

Educators also indicated being uncertain about the value of badges and how they could effectively use them in their teaching. Clearly, training and orientation for educators are key. An interesting two-pronged approach by Georgian College involved awarding badges as part of its faculty development program, as mentioned above, with the result of educators understanding the potential of Open Badges and subsequently exploring how they could integrate them into their teaching. A similar approach was suggested with institutional staff development—starting small could pique interest and spread information well before badges are integrated into academic and co-curricular programming.

A number of open badging pilots involved employer input and perspectives. Durham College reported that employers initially perceived badges to have little value; however, after learning what badges can represent, these employers urged their continuing education unit to use badging in future courses. A key recommendation from many partners was to include the Program Advisory Committee (PAC) in the development of badging systems, as these groups typically include employer and industry representatives.

Sandbox partners also considered the availability and value of the “badging endorsement” feature, which would enable any organization to endorse another organization’s badges. This feature, however, was only added to the CanCred Factory platform and presented to institutional partners two-thirds of the way into the pilot, which was too late for institutions to incorporate it into their projects. At the time, this additional feature was deemed to be a way of increasing badge credibility and creating strong partnership networks. Even though it was not used during the pilot, plans for industry and institutional endorsement of the feature are being considered for the future.

Badging Teams

The pilot evaluation period was a good opportunity to better understand what people need to be involved in the development of a robust badging system. All of our partners provided information on the staffing needs internal to the institution, and some mentioned what external perspectives were, and would be, valuable.

Many institution partners recommended that all members of the badging team have a good understanding of the value of open badges in addition to the process by which they are created, earned, issued, and shared. Clearly, having a badge orientation process for developers is a critical component of any project plan.

Internal badging teams emphasized a collaborative approach, with critical roles and various levels of participation from teaching and learning professions, information-technology professionals, learners, alumni, and external stakeholders who would have an interest in recognizing and endorsing the skills represented by the badge. Another recommendation was that this approach should be formalized by establishing a badging project steering group whose mandate would include a needs analysis, project oversight and management, and a communication plan that incorporates a regular reporting mechanism to academic leadership.

The involvement of the institutional branding office was also mentioned as a key step in establishing a badging system, depending on the culture of the institution. The visual design of a badge is reflective of the institution, whose brand and guidelines should be incorporated from the outset. Currently, retroactive application of design to badges that may have already been issued is problematic.

Partner institutions reported a need to ensure that any badge orientation process—for educators, for students, and for employers—be targeted to key stakeholders. They recommended integrating conversations about badges into PAC meetings as a method of increasing stakeholder engagement and understanding of how badges can benefit learners in the program.

Badge System Structure

The badges developed during the pilot were mainly linear in nature, with a couple of exceptions. Although Loyalist College did not issue any badges, the team developed a levelled co-curricular badge system that aligned with four tenets: learning, community, leadership, and self-reporting. The plan is for levelled badges to be awarded based on a points system: bronze star (100 points), silver star (200 points), and gold star (300 points). Fanshawe College also developed badging plans that clustered courses around a milestone achievement.

Badges were issued either automatically or after review of an application. Those that issued automatically were generally to earners who passed or completed a shorter, workshop-type learning experience. In these cases, the badges were issued by email. Those awarded upon application required the badging team to create the application steps in CanCred Factory and direct the learners to complete the form via an email link.

During the pilot process, institutions noted that conversations occurred concerning the creation of a shared badging taxonomy. A badging taxonomy fleshes out the potential uses and classifications of badges and provides a visual representation of similarities, differences, and connections across an institution’s badging system. Developing a shared badge taxonomy involves imposing consistent branded shapes or icons that may demonstrate rigour; having visual signifiers (e.g., colour, font, second shapes along perimeter) if the badges are part of a collection; and using a naming and issuing protocol.

A general discussion about the potential of a shared taxonomy was held by institution partners during an open badging webinar. Badge taxonomy resources were shared with and distributed through Basecamp. The badge taxonomy example from Mount Vernon Institute for Innovation was viewed as a helpful guide.

One sandbox partner, Georgian College, tied its badges to an existing framework: the Quality Matters Course Design Rubric. This framework was enabled by the “alignments” feature in CanCred Factory, which allows users to create or add their own frameworks to apply to their badges. Institutions engaged in the pilot agreed that the decision to align badges to existing standards and competencies can build credibility within the post-secondary education community and increase recognition of badges further afield. To this end, institutions in the pilot discussed using existing frameworks for employability skills (e.g., the Ontario Essential Employability Skills Framework) in connection with institutional objectives and graduate attributes (i.e., institutional learning outcomes).

User Experience

Feedback from project teams indicate that the CanCred platform is easy to use and that creating, designing, and issuing badges is fairly straightforward. Partners found that reports were easy to generate and functionality was intuitive and well designed. Two significant technical issues mentioned were the need to integrate with various learning management systems (LMS) and the difficulty in sharing badges on LinkedIn.

LMS integration was reported as the most logical solution for sustainability of any badging system. As this was not available for this pilot period, users developed a workaround with help from CanCred. However, this solution resulted in more steps, both for issuing the badge and for earners claiming the badge. At the time of this report, CanCred was close to completing the integration with Desire2Learn (D2L), which holds the biggest share of LMS use among Ontario post-secondary institutions.

Badge earners also reported challenges in trying to receive a badge, move it from an institution’s LMS to eCampusPassport, and then display it on LinkedIn. They reported there being too many steps, and many students eventually gave up, despite there being available documentation and guidance. Students suggested including a real-time online chat function site of support.

During the pilot period, LinkedIn changed how certifications are displayed on their web platforms. This resulted in a more manual process to add a badge to a user profile. CanCred Badge Factory provided sandbox partners with information on how to navigate this new process.

As part of their pilot project, the University of Waterloo also developed a tutorial on how to claim and share badges.

Other Challenges

In addition to the challenges mentioned above related to value statements, LMS integration, and sharing badges on LinkedIn, there were other “pain points” disclosed in the reports, which should be considered by those exploring the implementation of a badging system.

The main challenge is the additional work needed to develop a robust badging system. For many in the sandbox, this project was added to an already heavy workload, and many did not have time to make it a success. The technical, infrastructure, and human resources required to support implementation of a badging process are critical to ensure widespread understanding, endorsement, and adoption.

Another challenge highlighted as significant was the manual review process of badge applications. Such a review is needed to ensure that an earner has met all of the criteria and to uphold the quality and credibility of a badge. However, the time-intensive review process may be a barrier to widespread adoption by institutions. In response, some institutions considered implementing a peer review process, and awarding a “badge reviewer” badge to those learners (and earners) who qualify to review their peers’ work. However, at this early stage, it is paramount that the time needed to properly manage the badge application process is incorporated fairly into the job responsibilities of an individual or team. Doing this early on will improve chances of building a sustainable badging system.

Also noted was the time required to identify the key stakeholders both within and outside an institution. These key stakeholders are essential to establishing a working group early in the implementation of a badging system. When establishing working groups, institutions found that significant time was required to connect with representative stakeholders. When approaching stakeholders to participate in a badging project, leaders should be careful to accurately articulate the scope of involvement and time commitment required.

Finally, another hurdle of this pilot project was the five-week work stoppage throughout the Ontario college system during the first half of the evaluation period. During the fall of 2017, instructors were unavailable to participate in implementing badges in a meaningful way, and college staffing resources were compromised because of the strike.

Future Plans

Many of the sandbox partners have chosen to continue with their badging explorations. Those who have invested significant time to plan and/or pilot a small badging system feel that the work they have already completed provided a good introduction and set the stage for future success.

Future plans include additional badging within continuing education, faculty/professional development programs, and co-curricular activities. These plans are contingent upon integration with LMS as well as existing co-curricular student record systems. Partners specified plans to leverage existing partnerships with employers and obtain feedback from graduates entering the workforce to ensure that both curricular and co-curricular badges address skills needed, skills gaps identified, and new opportunities.

Some partners plan to create an online module that will be integrated into their LMS in order to build awareness of badges, their value, and potential incorporation into curriculum. This module could be turned on for courses that include badges.

As noted, endorsements will continue to be an area of great interest and a source of exploration for our partners. Third-party validation of a badge is viewed not only as a value-added opportunity to build credibility and create meaningful badge ecosystems, but also to help strengthen and grow existing collaborations. It is important to note that endorsements can be more formal (e.g., tied to an accreditation) or informal (e.g., meets with our approval or guiding principles and purpose). For further reading on the potential of endorsements, we recommended CanCred’s post by Don Presant to our partners .

The value of badges as prior learning assessment, particularly for the growing international student population, was also earmarked as an area of future exploration.

Support

A common thread among all the sandbox partners was that the support offered by CanCred and eCampusOntario was highly valued. Because there was a wide range of teams and varying levels of open badging awareness, the two organizations provided both group and one-on-one orientation webinars in addition to shared monthly badging clinics. The clinics provided a combination of onboarding and/or a tour of additional features added to the platform with featured speakers from well-established and recognized badging systems worldwide. The clinics were recorded and are available from the following playlist:

In addition, a Basecamp community was created to facilitate communication and sharing of documents and other resources. As awareness of the open badging sandbox pilot grew, we received interest from other Ontario post-secondary institutions. Individuals from additional institutions that had pursued CanCred Factory accounts outside of the sandbox project also joined our Basecamp, which grew to be a total of 31 members. This communication platform will continue after the conclusion of the project to foster conversations and development of open badging systems in Ontario. Anyone is welcome to join by contacting projects@ecampusontario.ca.

References

Badge Alliance Endorsement Working Group (2014). Badge endorsement: Getting started. Retrieved from https://docs.google.com/document/d/1VVf19d72KmGMh1ywrLe7HCKEOqGSI0WjvwfGN_8Q2M4

Discover Open Badges. (2018). Retrieved from https://openbadges.org/

Fong, J., Janzow, P., & Peck, K. (2016). Demographic shifts in educational demand and the rise of alternative credentials. University Professional and Continuing Education Association (UPCEA). Retrieved from https://upcea.edu/wp-content/uploads/2017/05/Demographic-Shifts-in-Educational-Demand-and-the-Rise-of-Alternative-Credentials.pdf

Hickey, D. T., Willis III, J., & Quick, J. (2015).  Where badges work better.  EDUCAUSE Learning Initiative ELI. Retrieved from https://library.educause.edu/~/media/files/library/2015/6/elib1503-pdf.pdf