Teaching and Learning Research with AI
As a teacher, you might decide to explore your teaching and student learning more deeply by engaging in the Scholarship of Teaching and Learning (SoTL). SoTL is a form of research, where a teacher undertakes a reflective and systematic investigation into their teaching and/or their students’ learning experiences. The goal of any SoTL study is to improve student learning and educational experiences (McKinney, 2007). Data can be gathered through various means, including surveys, classroom observations, examination of student work, interviews, focus groups, student evaluations, and more.
While AI is not a replacement for our insights, creativity, and decision-making as researchers, it can help us streamline and enhance parts of the research process.
5 Guiding Principles for SoTL Research
Peter Felten (2013) provides five principles of good practice for engaging in SoTL research. The guiding principles for SoTL research are:
- Inquiry into Student Learning: SoTL research is fundamentally about investigating student learning. This involves asking questions about how students learn, what they learn, and how teaching can improve learning.
- Grounded in Context: SoTL research is grounded in the specific context in which teaching and learning occur. This means that the research considers the unique characteristics of the students, the teacher, the discipline, and the institution.
- Methodologically Sound: SoTL research must be conducted using sound research methodologies. This ensures that the findings are reliable and valid.
- Conducted in Partnership with Students: SoTL research is often conducted in partnership with students. This entails inviting students to be active participants in the research process.
- Appropriately Public: SoTL research involves ‘going public’ and sharing findings with others through presentations, publications, resources, and many other forms of dissemination.
Using AI to Incorporate SoTL Principles
These principles serve as guideposts for developing and refining individual SoTL inquiries and larger SoTL initiatives. AI can enhance SoTL research by analyzing student learning patterns and helping to tailor research to specific contexts. It can help to ensure methodological rigour by using advanced algorithms to design robust studies, perform accurate data analysis, and validate findings.
AI might also be useful for facilitating student collaboration by providing interactive platforms where students can contribute their insights and feedback, making them active participants in the research process. Additionally, AI can streamline communication and coordination among researchers and students, enhancing the overall quality and relevance of the research.
See some ideas for using AI in the SoTL research process, along with some cautions when it comes to upholding the SoTL Guiding Principles.
Try This! Using AI in the SoTL Research Process
Use Copilot to enhance your SoTL research process. Copy and paste the prompt(s) below into Copilot. Replace the bracketed words with your course information. What ideas might be helpful for you?
Brainstorming SoTL Research Topics
Task: Identify research topics based on student feedback
Example: “Analyze the following dataset of student feedback, course evaluations, and survey responses. Identify common themes, recurring issues, and notable patterns related to teaching and learning experiences. Based on these insights, suggest potential research topics that address specific challenges or opportunities for improving my teaching and students’ learning. Ensure the suggested topics are relevant, actionable, and grounded in the context of the provided data. [insert student feedback]”
Task: Identify research topics based on recent research publications.
Example: “Analyze the most recent publications in Scholarship of Teaching and Learning journals. Identify emerging trends, recurring themes, and notable gaps in the literature. Based on these insights, suggest potential research topics that could inspire new investigations and contribute to the advancement of teaching and learning practices. Ensure the suggested topics are relevant, innovative, and grounded in the context of the identified trends and gaps.”
Generating SoTL Research Questions
Task: Generate a research question based on a chosen topic
Example: “Generate a research question based on the following topic within the Scholarship of Teaching and Learning (SoTL): [Insert Specific SoTL Topic Here]. The research question should be specific, clear, and focused on a particular aspect of the chosen SoTL topic. It should also be open-ended to allow for in-depth exploration and analysis.”
Task: Revise a research question based on existing research.
Example: “Review the following research question in the field of [Insert Field Here]: [Insert Research Question Here]. Based on existing research questions in this field, provide feedback on how to refine this question to ensure it is unique and impactful. The feedback should include suggestions for improving clarity, specificity, and relevance.”
Activity and Prompt Samples: Summarizing Existing literature
Task: Generate a list of relevant literature to support a SoTL study
Example: “Generate a list of relevant literature to support a study on the following topic within the Scholarship of Teaching and Learning (SoTL): [Insert Specific SoTL Topic Here]. The list should include peer-reviewed articles, books, and other credible sources that provide valuable insights and evidence related to the chosen topic.”
Task: Summarize existing literature on a specific topic
Example: “Summarize the existing literature on the following topic: [Insert Specific Topic Here]. The summary should include key findings, major themes, and significant studies or articles that have contributed to the understanding of this topic. Ensure the summary is concise and highlights the most important points.”
Notes
You can also specify summarize existing literature only in “SoTL journals” or specific discipline-based journals in your field, for example only “STEM education.”
Caution
When using AI to support a literature review, be careful about the accuracy and reliability of the information provided. AI can help summarize and synthesize large volumes of data, but it may not always distinguish between high-quality, peer-reviewed sources and less credible ones. Additionally, AI-generated content might miss nuanced insights or context that a human researcher would catch. Therefore, it’s crucial to cross-check AI-generated summaries with original sources, ensure the inclusion of diverse perspectives, and critically evaluate the relevance and credibility of the information. This approach helps maintain the integrity and depth of your literature review.
Activity and Prompt Samples: Choosing & Designing Data Collection Methods
- Task: Write a prompt to generate data collection options based on SoTL topic
- Example: “Suggest various data collection methods suitable for a study on the following topic within the Scholarship of Teaching and Learning (SoTL): [Insert Topic Here]. The suggestions should include both qualitative and quantitative methods, along with a brief explanation of how each method can be applied to the study and what evidence it can provide.”
- Task: Write a prompt to generate data collection options based on SoTL research question
- Example: “Provide an overview of data collection techniques that can be used to investigate the following research question within the Scholarship of Teaching and Learning (SoTL): [Insert Research Question Here]. Include examples of specific tools or instruments that can be used for each technique and discuss their advantages and limitations.”
Activity and Prompt Samples: Qualitative Data Analysis
- Task: Write a prompt to help develop a coding framework and data categorization
- Example: “Assist in coding the following qualitative data collected from [Insert Data Source Here, e.g., student reflections, classroom observations] on the topic of [Insert Specific SoTL Topic Here]. Suggest a coding framework and categorize the data accordingly. Highlight any significant trends or noteworthy observations.”
- Task: Write a prompt to analyze a data set and suggest common themes
- Example: “Analyze the following qualitative data collected from [Insert Data Source Here, e.g., student interviews, focus groups, open-ended survey responses] on the topic of [Insert Specific SoTL Topic Here]. Identify key themes, patterns, and insights that emerge from the data. Provide a summary of these findings.”
Caution
It’s important to be cautious about the accuracy and interpretation of the data. AI can help identify themes and patterns, but it may miss subtle nuances and context that a human analyst would catch. Additionally, AI might introduce biases based on the data it was trained on, potentially skewing the analysis. Be sure to cross-check AI-generated insights with manual analysis to ensure validity and reliability. Always maintain a critical perspective, verifying the AI’s findings against the original data and considering diverse viewpoints to avoid oversimplification and ensure a comprehensive understanding of the qualitative data.
Activity and Prompt Samples: Quantitative Data Analysis
- Task: Write a prompt for data summary and create data visualizations
- Example: “Analyze the provided dataset on student performance, summarize key statistics (mean, median, standard deviation), and create visualizations (histograms, box plots) to illustrate the distribution of scores.”
- Task: Write a prompt to perform correlation and regression analysis
- Example: “Perform a correlation analysis between student attendance and exam scores in the provided dataset. Additionally, conduct a linear regression analysis to determine the strength and nature of the relationship between these variables.”
Caution
AI can sometimes produce inaccurate or misleading results if the data you input is biased, incomplete, or improperly formatted. Additionally, AI-generated analyses might lack the nuanced understanding that human experts bring, potentially overlooking critical context or misinterpreting data trends. It’s essential to validate AI-generated insights with traditional statistical methods and expert review to ensure reliability and accuracy.
Activity & Prompt Samples: Sharing Research Findings
Task: Practice prompting to explore ideas for various forms of sharing research findings
Example: “Summarize the key findings of the provided SoTL research paper in a concise and engaging manner suitable for a general audience. Highlight the main results, their implications, and any recommendations for educators.”
Task: Write a prompt for generating engaging social media content to share research
Example: “Create a series of social media posts (Twitter, LinkedIn, Facebook) based on the findings of the following SoTL research [insert findings]. Each post should include a brief summary of a key finding, an eye-catching graphic or image, and a call to action for further reading or engagement
Task: Write a prompt for identifying publication outlets
Example: “Identify conferences, academic journals, and online platforms that are relevant to my Scholarship of Teaching and Learning (SoTL) research topic on [insert specific topic here]. Briefly describe each and explain why they are suitable for disseminating my research findings.”
Disclosing the Use of AI in Research
Incorporating AI into research brings numerous advantages but necessitates transparency and ethical considerations. As members of academic and professional communities, we are responsible for ensuring the integrity and reliability of our research processes and findings.
It is good practice to start getting used to disclosing your use of AI. The following guidelines, adapted from Resnik and Hosseini (2024) may help you to communicate the role of AI in your research effectively:
- Explicit mention: Clearly state the use of AI in the various sections of your research, for example literature reviews and methodologies. Consider describing how AI tools were used, including the specific tasks they performed, such as data analysis, image processing, hypothesis generation, condensing literature and so on.
- Detailed limitations: Throughout your research, explain any limitations or potential biases introduced by the AI tools. This includes the data collection phase, analysis, and interpretation of results, helping readers understand the context and reliability of your findings
- Acknowledge contributions: While AI tools should not be listed as authors, their contributions should be acknowledged in the acknowledgments section. Specify the software or algorithms used and their roles in different parts of the research process.
- Ethical training: Ensure all researchers involved in your project are trained in the ethical use of AI, including understanding its limitations and potential biases.
Currently, we are still learning more about the role of AI in research, but it is unlikely that our academic/professional ethical standards will change. There are many calls for developing new guidance for the appropriate use of AI in research (Resnik& Hosseini, 2024). Looking ahead, SoTL researchers must address ethical concerns, such as data privacy, bias, and content quality. Adopting AI will broaden research possibilities, but it also demands continuous learning and adaptation to new ethical standards and best practices.