Evaluating A.I. Tools for Use in Education: A Rubric for Teachers and Students

Laura Morrison; Janette Hughes; and Chris D. Craig

Themes: Assessment, Ethical challenges in using AI, Lesson planning, Teaching Strategies
Audience & Subject: General (All Grades)

Introduction

Artificial Intelligence (AI), as a tool, is already impacting student learning experiences, and educators will benefit from frameworks that address AI literacy to help students successfully construct their learning experiences. However, a critical approach or framework—an underlying structure to develop something conceptual or tactile that guides how something can be done—for AI in education remains underexplored despite its importance and societal impacts (Ng et al., 2021). Recognizing that progress is occurring and students’ technical skills are improving, unified Canadian AI literacy frameworks still need to align with existing structures and standards, such as the Ontario Ministry of Education’s (OME) promotion of global competencies and ISTE for students. Our team of researchers synthesized AI ethics frameworks developed by key stakeholders in education, government, business and other global organizations to develop an AI ethics framework for education (Government of Ontario, 2023; UNESCO, 2021; IBM, 2024; the World Economic Forum, 2021; Google, n.d.; Microsoft, 2023; The White House, 2022; TEACH online, 2023). This framework was then adapted into a rubric that teachers and students could use in the classroom to analyze various AI tools.

General Guidelines

Application of the rubric in the K-12 classroom includes:

  • Teachers using it to analyze AI tools for integration into the teaching and learning process;
  • Students using the rubric to analyze various AI tools as part of the English Media Literacy strand; and
  • Teachers/students using the rubric in other courses that plan to incorporate a critical examination of AI tools in society.

Activity: Critical Analysis of Adobe Firefly

Overview

Adobe Firefly is a generative AI program that enables users to generate images through descriptive prompts. As with all LLMs, the more specific a prompt is, the more accurate the generated representation will be (Mesko, 2023). Like ChatGPT, Adobe Firefly uses a large language model (LLM) to “understand” the meaning of the words in a prompt. The Adobe website explains that Firefly training involves the use of Adobe Stock images along with openly licensed and public domain content (Adobe, 2023).

Description

This activity has participants complete a critical analysis of the tool Adobe Firefly using the AI Ethical Framework for Education rubric. For this, students use the main categories from the rubric to drive their investigation and analysis of the AI tool: universality, empowerment, transparency, safety and accountability. As the students do a deep dive into universality, empowerment, transparency, safety and accountability, they use the 7-point Likert scale on the rubric to give the tool a final score in each category. Besides their Likert scale rating, there is space for students to defend their rating with evidence from their research in the tool.

Key Benefits

Key benefits include the development of participants’ knowledge of AI image generation tools, specifically Adobe Firefly; the development of participants’ awareness of key ethical issues connected to AI in general and image generation tools like Adobe Firefly more specifically; and an opportunity to test the AI in Education Ethical Framework rubric for potential use in other settings, like the K-12 classroom.

Possible Challenges

Some challenges include participant hesitancy to register for an Adobe Firefly account and the risks associated with using AI tools in general, such as privacy concerns, data collection concerns, etc. One way to potentially address this challenge is to create mock accounts for participants to use before the session to sidestep some of these concerns. In addition, it may be necessary to explain some of the basics of how AI image generators are trained, the ethical implications of using them (even with mock accounts) in terms of training and considerations for uploading/sharing personal information or images in the AI system.

Support Materials

For this activity, you will need to use the AI in Education Ethics Framework Rubric – please click on the following link to open a forced copy of the rubric. You can download the rubric to your computer to edit: https://tinyurl.com/yck837rv. You will also need access to Adobe Firefly: https://www.adobe.com/products/firefly.html.

Resources

 

References

Adobe. (2023, September 13). Adobe releases new Firefly Generative AI models and web app; integrates Firefly into Creative Cloud and Adobe Express. Adobe News. Retrieved April 2024, from https://news.adobe.com/news/news-details/2023/Adobe-Releases-New-Firefly-Generative-AI-Models-and-Web-App-Integrates-Firefly-Into-Creative-Cloud-and-Adobe-Express/default.aspx

Google. (2024). A guide to AI in education. Google for Education. https://services.google.com/fh/files/misc/gfe_guide_to_ai_in_education.pdf

Government of Ontario (2023). Ontario’s trustworthy artificial intelligence (AI) framework. Ontario. https://www.ontario.ca/page/ontarios-trustworthy-artificial-intelligence-ai-framework

IBM. (2024). Foundation models: Opportunities, risks, and mitigations. IBM AI Ethics Board. Retrieved April 2024, from https://www.ibm.com/impact/ai-ethics

Mesko, B. (2023). Prompt engineering as an important emerging skill for medical professionals: Tutorial. Journal of Medical Internet Research, 25, 1-6. https://doi.org/10.2196/50638

Microsoft. (2023). Alignment of our efforts with the White House voluntary AI commitments. Microsoft. https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2023/07/white-house-camparison-chart-EDIT-1.jpg

Ng, D., Leung, J., Chu, S. & Qiao, M. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, Article #100041. https://doi.org/10.1016/j.caeai.2021.100041

TEACH Online (2023, July 24). A framework for the deployment of AI in Canada’s post-secondary education system. Contact North. Retrieved April 2024, from https://teachonline.ca/tools-trends/framework-deployment-ai-canadas-post-secondary-education-system

UNESCO. (2021). Key facts: UNESCO’s recommendation on the ethics of artificial intelligence. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000385082

White House. (2022, October). Blueprint for an AI bill of rights: Making automated systems work for the American people. The White House. https://www.whitehouse.gov/ostp/ai-bill-of-rights/

World Economic Forum. (2021, June 23). 9 ethical AI principles for organizations to follow. World Economic Forum. https://www.weforum.org/agenda/2021/06/ethical-principles-for-ai/


About the authors

Laura Morrison, Ed.D., is an Assistant Professor in the Faculty of Education at Ontario Tech U. Her work focuses on promising practices related to online education, inclusive maker pedagogies and critically digital literacies. Laura’s current grant-funded research explores the use of culturally sustaining maker pedagogies to increase engagement and learning for students typically underrepresented in STEM.

Dr. Janette Hughes is a Canada Research Chair in Technology and Pedagogy and Professor at Ontario Tech University. She is the recipient of multiple research and teaching awards and research grants. She is widely published and author of The Digital Principal, a guide for school administrators interested in promoting technology-rich learning environments for students and teachers. Dr. Hughes is a prolific author and presenter, sharing her work nationally and internationally in prestigious scholarly and professional journals, keynote talks, and conferences. Dr. Hughes is routinely contacted by school districts, Ministry personnel and industry partners to consult on a variety of topics, including online teaching and learning, equity issues in ed tech, creating innovative learning environments, establishing Makerspaces in schools, shifting pedagogies in a digital era and how to foster the development of global (21st century) skills and competencies in K-12 and higher education.

Chris D. Craig is passionate about education and has diverse life experiences outside of academia, which include the arts, business, construction, entertainment, and health. With a focus on adult education, he aims to establish and promote actionable insight for educators and students alike with the vision of supporting learner self-concept: Who they are in the context of the environment. Chris is an Ed.D. candidate and received an M.Ed., B.A., A.S.-PE., and certificates in project management, art, and entrepreneurship. Outside of education, you can find him happy to connect over coffee and discuss human nature, graphic novels, strength and conditioning, and random TV or movies.

License

Share This Book