Evaluating A.I. Tools for Use in Education: A Rubric for Teachers and Students
Laura Morrison; Janette Hughes; and Chris D. Craig
Themes: Assessment, Ethical challenges in using AI, Lesson planning, Teaching Strategies
Audience & Subject: General (All Grades)
Introduction
Artificial Intelligence (AI), as a tool, is already impacting student learning experiences, and educators will benefit from frameworks that address AI literacy to help students successfully construct their learning experiences. However, a critical approach or framework—an underlying structure to develop something conceptual or tactile that guides how something can be done—for AI in education remains underexplored despite its importance and societal impacts (Ng et al., 2021). Recognizing that progress is occurring and students’ technical skills are improving, unified Canadian AI literacy frameworks still need to align with existing structures and standards, such as the Ontario Ministry of Education’s (OME) promotion of global competencies and ISTE for students. Our team of researchers synthesized AI ethics frameworks developed by key stakeholders in education, government, business and other global organizations to develop an AI ethics framework for education (Government of Ontario, 2023; UNESCO, 2021; IBM, 2024; the World Economic Forum, 2021; Google, n.d.; Microsoft, 2023; The White House, 2022; TEACH online, 2023). This framework was then adapted into a rubric that teachers and students could use in the classroom to analyze various AI tools.
General Guidelines
Application of the rubric in the K-12 classroom includes:
- Teachers using it to analyze AI tools for integration into the teaching and learning process;
- Students using the rubric to analyze various AI tools as part of the English Media Literacy strand; and
- Teachers/students using the rubric in other courses that plan to incorporate a critical examination of AI tools in society.
Activity: Critical Analysis of Adobe Firefly
Overview
Adobe Firefly is a generative AI program that enables users to generate images through descriptive prompts. As with all LLMs, the more specific a prompt is, the more accurate the generated representation will be (Mesko, 2023). Like ChatGPT, Adobe Firefly uses a large language model (LLM) to “understand” the meaning of the words in a prompt. The Adobe website explains that Firefly training involves the use of Adobe Stock images along with openly licensed and public domain content (Adobe, 2023).
Description
This activity has participants complete a critical analysis of the tool Adobe Firefly using the AI Ethical Framework for Education rubric. For this, students use the main categories from the rubric to drive their investigation and analysis of the AI tool: universality, empowerment, transparency, safety and accountability. As the students do a deep dive into universality, empowerment, transparency, safety and accountability, they use the 7-point Likert scale on the rubric to give the tool a final score in each category. Besides their Likert scale rating, there is space for students to defend their rating with evidence from their research in the tool.
Key Benefits
Key benefits include the development of participants’ knowledge of AI image generation tools, specifically Adobe Firefly; the development of participants’ awareness of key ethical issues connected to AI in general and image generation tools like Adobe Firefly more specifically; and an opportunity to test the AI in Education Ethical Framework rubric for potential use in other settings, like the K-12 classroom.
Possible Challenges
Some challenges include participant hesitancy to register for an Adobe Firefly account and the risks associated with using AI tools in general, such as privacy concerns, data collection concerns, etc. One way to potentially address this challenge is to create mock accounts for participants to use before the session to sidestep some of these concerns. In addition, it may be necessary to explain some of the basics of how AI image generators are trained, the ethical implications of using them (even with mock accounts) in terms of training and considerations for uploading/sharing personal information or images in the AI system.
Support Materials
For this activity, you will need to use the AI in Education Ethics Framework Rubric – please click on the following link to open a forced copy of the rubric. You can download the rubric to your computer to edit: https://tinyurl.com/yck837rv. You will also need access to Adobe Firefly: https://www.adobe.com/products/firefly.html.
Resources
- World Economic Forum (2023, June 5). Why ethical AI requires a future-ready and inclusive education system. World Economic Forum. https://www.weforum.org/agenda/2023/06/ethical-ai-future-ready-inclusive-education-system/
- Elgersma, C. (2024, March 6). Here’s what educators can do as artificial intelligence evolves. Common Sense Education. https://www.commonsense.org/education/articles/chatgpt-and-beyond-how-to-handle-ai-in-schools
- aiEDU: https://www.aiedu.org/
- 300+ AI Tools from Julie Smith – Webster Univ: https://docs.google.com/spreadsheets/d/1DvAX-MajOOLKinUZXClq9xLinS0Pl4W6vQb3me5iEog/edit?usp=sharing
References
Adobe. (2023, September 13). Adobe releases new Firefly Generative AI models and web app; integrates Firefly into Creative Cloud and Adobe Express. Adobe News. Retrieved April 2024, from https://news.adobe.com/news/news-details/2023/Adobe-Releases-New-Firefly-Generative-AI-Models-and-Web-App-Integrates-Firefly-Into-Creative-Cloud-and-Adobe-Express/default.aspx
Google. (2024). A guide to AI in education. Google for Education. https://services.google.com/fh/files/misc/gfe_guide_to_ai_in_education.pdf
Government of Ontario (2023). Ontario’s trustworthy artificial intelligence (AI) framework. Ontario. https://www.ontario.ca/page/ontarios-trustworthy-artificial-intelligence-ai-framework
IBM. (2024). Foundation models: Opportunities, risks, and mitigations. IBM AI Ethics Board. Retrieved April 2024, from https://www.ibm.com/impact/ai-ethics
Mesko, B. (2023). Prompt engineering as an important emerging skill for medical professionals: Tutorial. Journal of Medical Internet Research, 25, 1-6. https://doi.org/10.2196/50638
Microsoft. (2023). Alignment of our efforts with the White House voluntary AI commitments. Microsoft. https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2023/07/white-house-camparison-chart-EDIT-1.jpg
Ng, D., Leung, J., Chu, S. & Qiao, M. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, Article #100041. https://doi.org/10.1016/j.caeai.2021.100041
TEACH Online (2023, July 24). A framework for the deployment of AI in Canada’s post-secondary education system. Contact North. Retrieved April 2024, from https://teachonline.ca/tools-trends/framework-deployment-ai-canadas-post-secondary-education-system
UNESCO. (2021). Key facts: UNESCO’s recommendation on the ethics of artificial intelligence. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000385082
White House. (2022, October). Blueprint for an AI bill of rights: Making automated systems work for the American people. The White House. https://www.whitehouse.gov/ostp/ai-bill-of-rights/
World Economic Forum. (2021, June 23). 9 ethical AI principles for organizations to follow. World Economic Forum. https://www.weforum.org/agenda/2021/06/ethical-principles-for-ai/