University of Waterloo

Project Description

Our project goal was to develop strategies for enhancing student academic integrity knowledge using interactive scenarios through open access mobile technology, culminating with the achievement of a certificate and a digital badge to recognize completion. The vision of our digital badging research project aimed to explore the best strategies, from a student user perspective, for accessing and understanding student interest in using digital badges.

We carried forward our project goals and vision during the digital badging development process. Our initial badging pilot project was scaled down due to the seasonal timing of implementation of the testing process when students had completed schooling over the summer. We found the time it took for the quality assurance process for the IntegrityMatters app was extended due to multiple revisions before we implemented the badge testing into the final version of the app. We intended to ensure the bugs were removed from the app before adding the badge credentials through an APK file. We successfully tested the badging system for quality assurance and awarded a digital badge for completion of the mobile lessons on academic integrity at a passing rate of 75%.

Badging Team Description

The digital badge team comprised five members: the project technical lead, user experience/digital badge designer, principal researcher, and two staff from the Office of Academic Integrity.

Two separate versions of our app, IntegrityMatters, were released to the iTunes Store and the Google Play Store in three separate language: English, French, and Mandarin.

Link to the App

• iOS: https://itunes.apple.com/us/app/integritymatters/id1355112345?mt=8
• Android: https://play.google.com/store/apps/details?id=uwai.dev.uwai&hl=en
• ID and Password: No (the IntegrityMatters app is open access)

Special Instructions: After selecting the language to interact with the application, you’ll be presented with a login page. Scroll down to the bottom of the page and you’ll see a button labelled “Login as Guest.” Select that, and you’ll then be able to interact with the application in full.

Our team developed an academic integrity digital badge, in addition to developing our own badge system design workflow (see Appendix B) and a digital badge training video.

Study participants (undergraduate university students) who successfully completed the academic integrity IntegrityMatters app lessons with an overall score of 75% were invited to participate in the testing of our digital badge process. Study participants were asked to claim their badge at the CanCred.ca Passport site and export it to their social media profiles on LinkedIn, Twitter, Facebook or Google+ profile. In addition, they were required to complete a digital badging questionnaire (see Appendix B) to report on their experience. Some of the questions were scaling questions (1–5) where participants could add comments in text if they wanted to expand on their answers in the rating scale. The study results will be disseminated at conferences.

Badging System Structure

The Academic Integrity Digital Badge is awarded for successful completion of all the IntegrityMatters six modules with 75% or higher. Each module focuses on scenarios involving diverse aspects of student academic life (cultural differences and expectations, physical stress, peer pressure, time constraints, etc.). Learners who successfully complete all of the modules are awarded a badge validating their competency in this functional area of academic integrity knowledge. Learners may choose whether or not to claim badges earned. Additionally, learners may choose to make their badges publicly viewable. When a badge is earned, an email is sent to learners who can then claim the badge at the CanCred.ca Factory Passport site (https://passport.cancred.ca).

Badge earners gained basic digital academic integrity knowledge; they are aware of the impact and outcomes of academic integrity in an academic setting. According to the International Center for Academic Integrity, academic integrity is “a commitment, even in the face of adversity, to six fundamental values: honesty, trust, fairness, respect, responsibility, and courage” (Fundamental Values Project, 2014). Learners experience academic integrity knowledge of the six values by completing interactive scenarios through open access mobile technology.

User Experience

Due to the unavailability of large student population in the midst of summer, we scaled down our project, selecting and awarding 77 mobile academic integrity digital badges. Forty users accepted the badge and completed the pilot testing. We tested the digital badging system and it was formally used for the research project.

Generally, the pilot users were satisfied with the platform and found it very easy to use and share on different social media. However, users reported some glitches where they were unable to directly input their badge to their LinkedIn account without copying the licence number and link from the badge site to LinkedIn. To address some of the technical challenges, the project team developed a training video to guide users to claim their badge for various social media. Users had access to technical support, as necessary, during the pilot testing phase. The project team felt it was necessary to support Unicode, especially for Asian languages, including Chinese and Japanese. This type of support could facilitate the issuing of badge to multiple language learners.

The project team is very pleased and grateful for the support from eCampusOntario, especially Don Presant for his willingness to share his expertise. We also appreciated the orientation sessions and virtual badging clinics, hosted by Joanne Kehoe, as a place to address questions and to talk with other colleges and universities who were testing digital badge implementation processes.

Value

We developed a 10-minute online questionnaire for users to complete to evaluate their badging experience). Qualitative findings from this report included some helpful feedback from users. Responses from 33 participants who completed the badging questionnaire varied. Most were pleased to be able to add the badges to their social media platforms, especially on LinkedIn.

• Seventy-nine percent of users responded in Question 1 they were satisfied with their digital badging options. While some users found the badging process relatively simple to use, others were not sure their badge was placed in the social media platform of their choice. Some users indicated they experienced technical challenges with their mobile phones and were going to double check on their laptop computers to ensure the badge transfer was completed.
• In Question 2, a number of users indicated they were new to the badging experience, while others found the process a novel alternative to a certificate. One user comment indicated that, “it allows employers to see that we are certified individuals, and the URL provides a level of authentication to the badge.” Twenty-seven percent of participants indicated they felt that badging was developing breakthrough technology while another 27% disagreed with this statement.
• In Question 3, almost seventy-one percent of student users indicated the badging process worked for them without any bugs, delays or errors. Some commented the training video was helpful for them to follow the steps to obtain their badge. One person indicated they had to re-enter their email address to claim their badge.
• In Question 4, over 87% agreed with the statement that digital badging was easy to use. Comments included, “Very simple and easy to follow. It’s intuitive”; “The process is straightforward”; and “Adding it to LinkedIn was easy, especially with the YouTube video.” Alternative feedback indicated that setting up an account “was a tedious process” for them.
• The responses to the question of whether digital badging is a useful process in Question 5 were mixed, with some participants agreeing (close to 30%), some disagreeing (27%), and some being neutral (33.3%). The comments shared that help explain the positive stance included, “It allows potential employers to see my badge without having to present a physical one to them,” while others were unsure about the practicality of badges and whether others would recognize them, saying, “Not sure how credible an online badge is and whether it is easy to copy them.”
• When asked about the value of staying connected to people important to them (Question 6), 42.5% of respondents indicated their university connection mattered to them; for example, “This badge is mainly for lab instructors/TAs/profs to show that a student understands the policies surrounding plagiarism,” while almost 27.8% said that potential employers would be interested in seeing evidence of badging “can help employers identify more competent individuals.” Sixteen percent indicated that badges wouldn’t enhance connectivity to people: “Doesn’t really aid in maintaining connections. Potentially aids in expressing your qualifications and achievements.”
• Finally, study participants were asked how they would recommend enhancing digital badging in Question 7. Suggestions ranged from “Higher resolution, enhance image quality” to “Remove registration” to “Auto-filling the field so that you do not need to copy each field of information in order to share it on social media” to “Market badges better, make badging more prevalent.” Another suggestion included, “Make the badge public automatically, unless you specifically want to set it as private. Kind of like how Facebook posts work.” A helpful suggestion was “Integrate it into LEARN [learning management system] so that students starting a new course/term will need to obtain the badge before submitting assignments. This would replace integrity quizzes the profs have for each course and can be done all at once with the badge as proof of agreement, completion, etc.” Others indicated additional security for badging, such as, “I would add some level of further authentication in case the badge was provided to the wrong individual.” Another commented on direct integration with social media: “Having direct links to share via Twitter, LinkedIn, Facebook, etc., if possible.” Students often felt that badging was unfamiliar to schools and employers: “It needs to become a more widespread practice in order for it to be effective for students in the job search process.” A few students indicated they would prefer to print the badge or have a tangible representation of completion of their work while others felt the badging process worked well.

Overall, participant feedback provided a range of responses, and the answers were very helpful to the research team. A student indicated that, “Badging was relatively simple to set up; it keeps up with the modern technology and is heavily used for connecting and self-marketing. A common response was that students wanted their badges integrated with the school’s learning management system (LMS) to show their profs, TAs, and other students of the work they had completed. These responses encouraged us to seek funding to help support continuation of the digital badging in our IntegrityMatters project as we work to streamline our project into our learning management system.

In addition to this digital badging report, we made multiple conference presentations. We published a paper entitled, “Academic Integrity Matters: Successful Learning with Mobile Technology,” co-authored by Alice Schmidt Hanbidge, Tony Tin, and Herbert Tsang, which was presented at the 21st International Conference on Interactive Collaborative Learning, Greece conference in September 2018. In our paper, we included this acknowledgement:

Acknowledgements: The authors wish to acknowledge research funding support from eCampusOntario. Also, the authors acknowledge support from the University of Waterloo, Renison University College, and Trinity Western University.

Challenges

We experienced and addressed a number of challenges during our digital badge development and testing process:

1. Interoperability with LinkedIn and other social media: the badging system needs to be more intuitive in terms of integration and interoperability.
2. System support for Asian and other languages is needed.
3. The subscription is not affordable for some smaller educational institutions. An affordable business model is necessary to sustain long-term engagement with digital badging.
4. More marketing and promotion would be helpful. People, our university student population, faculty, and staff were not knowledgeable about digital badging or had limited understanding of the benefits of badging credentialling.
5. Intrinsic and extrinsic motivators would be helpful to encourage undergraduate students to be lifelong learners with badging credentialling. If badge rewards were no longer desirable, student motivation could be reduced.
6. Digital badging processes need more interactive features, such as QR or AR code feature to enhance accessibility, visibility, and visual appeal.
7. More consideration should be given to promote proper academic behaviour that aligns with the learners, not the issuers.
8. The badge earner’s performance is not directly observed so there could be some difficulty in making sure that the badge is awarded to the correct person who completed the assignment or met the specific badging criteria.
9. More research and evidence-based studies are needed to determine the effectiveness of using open digital badges in academic settings and what tangible positive learning outcomes it enhances.

Future Plans

This small-scale pilot was well received by the students and the faculty. In addition, the survey also indicated that, overall, students are satisfied with the badge system. We have already established well-defined badging criteria and a workflow system for granting digital badges upon successful completion of the lesson modules using mobile learning. We appreciated the opportunity to learn more about e-badges and to develop a prototype badge for testing with our academic integrity app. Thank you for welcoming us to your Basecamp and for hosting the badge clinics.

The American Councils for International Education (https://www.americancouncils.org ) has expressed interest in our IntegrityMatters app. We are currently exploring how to best partner with them to endorse our app and our digital badge. A partnership will bring the IntegrityMatters app to its broad networks in higher, secondary, and primary institutions around the world.

We created a collaborative working group together with Kwantlan Polytechnic Institute in British Columbia to support them in translating our app into Punjabi and to create additional academic integrity scenarios to add to our digital databank of case scenarios.

Digital badging holds great potential for our institutional use. We secured funding through our eCampusOntario grant to renew our badge subscription for the upcoming academic year. It is anticipated that we will make a good use of this digital tool to promote and educate our users about the value of academic integrity and benefit of open digital badges.

Lessons Learned

Our research team wishes to sincerely thank eCampusOntario for continued support and encouragement in our project! Thank you for the opportunity to participate in the digital badging process.