Open Assessment

The open education movement promotes a way of thinking about education that enhances teaching and learning by incorporating attributes such as sharing, collaboration, adaptation, free access and reuse of information and spaces for educational interaction, etc. (Downes, 2013; Knox, 2013). Open assessment of learning is understood as an open educational practice and can be defined as:

“The process of learning verification and feedback that takes place collaboratively, mediated by free-access tools in which teachers produce or adapt assessment resources and students adapt and reshape these resources for the purpose of generating for themselves an assessment that meets their personal needs, learning styles and context.” (Chiappe, 2012, p. 10).

Recall that open assessment implies that teachers design learning tasks that foster not only teacher assessment, but also peer assessment. This approach emphasizes reflective practices and improved learning outcomes. The use of technology implies that teachers should use different technologies to facilitate the OEP within a course. These technologies include OER authoring tools, OER repositories, social networks and collaborative editing tools — and it is essential to adjust assessments to support OEP. Students are assessed based upon collaborative, renewable assignments that result in the creation of supplemental learning resources that all students can benefit from (and for future students to build upon). In some cases, open class forums have been established to create a space for students to discuss the feedback and answers to assessments and learn from each other. The remainder of this module focuses on the practice of designing renewable assignments for formative assessment (as shown in Figure 4, below) in open learning experiences.

Figure 4.

Framework for Designing OEP-based cCurses.

Note: This figure highlights the open assessment tip of the OEP framework. It focuses on the teacher-OER interaction through formative assessment and the practice of creating renewable assignments.

Adapted from A Case Study of Applying Open Educational Practices in Higher Education during COVID-19: Impacts on Learning Motivation and Perceptions by Zhang, et. al, and is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY 4.0) licence.

What are Renewable Assignments (RA)?

Renewable assignments (non-disposable assignments) are defined as “tasks in which students compile and openly publish their work so that the assignment outcome is inherently valuable to the community” (Chen, 2018; Wiley & Hilton, 2018). A renewable assignment is one that:“What if we changed these ‘disposable assignments’ into activities which actually added value to the world? Then students and faculty might feel different about the time and effort they invested in them.”
— (Wiley, 2013, para 4).

• students are asked to engage in — as part of an organized course;
• promotes student learning — through the completion of the assignment;
• affords assessment of students’ learning of course objectives;
• adds value beyond students’ own learning — because students are invited to openly license and publicly share their digital creations.

Explore the “Open Pedagogy as Open Student Projects” chapter of Open Pedagogy Approaches, for examples of renewable assignments — many of which contain descriptions of an original assignment, the process of redesign, the final product and additional examples of rubrics and other assessment tools.

Five Principles for Designing Renewable Assignments

1. Fundamentally involve information collaboration and exchange.
2. As forms of responsive and responsible pedagogy, RA involve communication opportunities for revision, creativity, modifying key terms and objectives, etc.
3. Produce learning through co-operative critique.
4. The resulting product or practice is invited to be shared outside the teacher–student dimension of the OEP framework, creating opportunities for communal access of the RA as an information resource. The resulting content and its benefits extend beyond the classroom.
5. Opportunities for innovation with the goal of producing something unique, not replicate something within an existing framework.

How can I Implement Renewable Assignments? What Does That Look Like?

The following five-step “Renewable Assignment Design Framework” (as shown in Figure 5, below) was developed by Stacy Katz and Jennifer Van Allen, Ed.D. from Lehman College, CUNY, to provide a process for assignment redesign — that includes important considerations required when converting a traditional authentic assessment into a renewable assessment — and to map-out what that looked like not only for themselves, but for others interested in developing their open pedagogical practices.

Figure 5.

Renewable Assignment Design Framework

Katz, Stacy & Van Allen, Jennifer. Open Pedagogy Approaches. Milne Library. (CC BY 4.0 International).

Step 1: Analyze and Classify Current Assignment

Redesigning an assessment begins with a reflective analysis of the current assessment. Stemming from Wiley and Hilton’s (2018) four-part test for categorizing an assignment as disposable, authentic, constructive or renewable, the following questions are important to consider before beginning any form of redesign:

1. Are students asked to create new digital artifacts (essays, poems, videos, songs, etc.) or revise/remix existing OER?
2. Does the new artifact have value beyond supporting the learning of its author?
3. Are students invited to publicly share their new artifacts or revised/remixed OER?
4. Are students invited to openly license their new artifacts or revised/remixed OER? (Wiley & Hilton, 2018)

It is important that the outcomes and goals of the assessment are clear and fully understood before moving on to the next step.

Step 2: Consider Meaningful OER Contributions

After fully analyzing and classifying the assignment, it is important to consider how it can contribute to knowledge within the field of the assessment. Reviewing examples of other renewable assessments is a beneficial activity to understand OER contributions can be meaningful within a discipline or a broader community. While some of the examples (accessed through the links below) contribute to course development through supporting the school community, others add to the discipline by developing open resources on the topic.

• Making Sense of Open Education Module
• Open Pedagogy Approaches
• Open Pedagogy – A Guide to Making Open Textbooks with Students
• Why have students answer questions when they can write them?

Step 3: Select Tools and Repositories

There is a growing, ever-changing list of OER repositories available to search for existing OER that one can reuse, modify and adapt to suit the context of a particular assessment. Each repository features the ability to conduct a search by using key terms, subject matter, type of resource, etc. When selecting an OER, consider institutional access to editing and adaptation tools, authoring features provided in specific tools, the students’ digital literacy skills and the time that faculty are willing to devote to developing students’ digital literacy skills, understanding of the tool and understanding of OER within the course. With these factors in mind, the collaborative partnership explores the tools together to select one that meets the needs of the assignment, reaches the intended audience of the contribution and will be manageable by the instructor and students within the course.

Referatories (Super OER Search Engines)

• OASIS (Openly Available Sources Integrated Search):  https://oasis.geneseo.edu/
• Mason OER Metafinder (MOM):  https://oer.deepwebaccess.com/oer/desktop/en/search.html

Repositories

• eCampusOntario Open Library: https://openlibrary.ecampusontario.ca
• eCampusontario H5P Studio:  https://h5pstudio.ecampusontario.ca/
• OER Commons: https://www.oercommons.org/
• Creative Commons: https://search.creativecommons.org/
• Merlot: http://www.merlotx.org/find/textbooks.html
• MIT Open Courseware:  https://ocw.mit.edu/educator/
• LibreTexts: https://libretexts.org/
• Public Domain Review:  https://publicdomainreview.org

When exploring and evaluating repositories and tools, it is important to consider and ensure the intended audience has access. There are a variety of evaluation frameworks that can be used to determine if a resource will meet the outcomes of the assessment. Librarians or instructional designers may be able to help work through the process of technology selection and suggest appropriate tools.

There are several attributes to consider when evaluating OER. The frameworks below provide some guidance and resources to help.

Step 5: Finalize and Reclassify the Assignment

The next step of the process is to create a rubric that is logical and easy for students to understand. A partnership with a librarian or faculty developer is incredibly valuable at this stage of the design framework as they are able to review the assignment and apply a different lens to it. Reclassifying the assignment to ensure it meets the criteria of a renewable assignment may result in reflective engagement about the value of the assignment to the field, the effectiveness of the tools utilized, and the match between the assignment’s learning outcomes and the learning goals of the course.

Adapted from Open Pedagogy Approaches by Stacy Katz and Jennifer Van Allen and is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

The Importance of Feedback for Formative Open Assessment

Feedback  is emphasized as an enabling factor that should be considered beyond just quantification or measurement of learning. Several literature reviews on this subject (Debuse, Lawley, & Shibl, 2007; Gipps, 2005; Irons, 2007; Jordan & Mitchell, 2009; Li, Liu, & Steckelberg, 2010; Van den Berg, Admiraal, & Pilot, 2006; Webb, 2005) provide extensive details about the educational importance to students of receiving timely and relevant information about their learning, the perspective of automated systems, the possibilities of tailored and timely feedback and its importance to maintain adequate levels of student motivation — as well as the relevance of peer feedback.

Feedback should also consider the emotional aspects of the student because this increases motivation, develops new and better thinking and learning skills and reduces student dropout rates. Here arises the question of whether or not the teacher or tutor will have the skills and knowledge to achieve such monitoring and feedback in a highly digital context. Some authors state that it is possible, but qualify that it requires a great deal of teacher commitment, time and training (Hatziapostolou & Paraskakis, 2010).

Open assessment seeks to improve the participation of the learner as a key participant in the learning process — a peer learner — encouraging commitment to learning and increasing the validity and reliability of assessment activities through the implementation of actions and ongoing formative feedback. Feedback from collaborators has the potential to introduce different and valuable information that provides varying perspectives about the content.

References

Chiappe, A. (2012). Prácticas educativas abiertas como factor de innovación educativa con TIC. Boletín REDIPE, 818(1), 6–12.

Chiappe, A., Pinto, R. A., & Arias, V. (2016). Open Assessment of Learning: A Meta-Synthesis. The International Review of Research in Open and Distributed Learning, 17(6). https://doi.org/10.19173/irrodl.v17i6.2846Debuse, J., Lawley, M., & Shibl, R. (2007). The Implementation of an Automated Assessment Feedback and Quality Assurance System for ICT Courses. Journal of Information Systems Education, 18(4), 491–502.

Downes, S. (2013). The role of open educational resources in personal learning. In R. McGreal, W. Kinuthia, & S. Marshall (Eds.), Open educational resources: Innovation, research and practice, (pp. 207-221). Vancouver: Commonwealth of Learning and Athabasca University.

Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351.

Gipps, C. V. (2005). What is the role for ICT-based assessment in universities? Studies in Higher Education, 30(2), 171–180.

Hatziapostolou, T., & Paraskakis, I. (2010). Enhancing the Impact of Formative Feedback on Student Learning through an Online Feedback System. Electronic Journal of E-Learning, 8(2), 111–122.

Jordan, S., & Mitchell, T. (2009). e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40(2), 371–385.

Knox, J. (2013). The limitations of access alone: Moving towards open processes in education technology. Open Praxis, 5(1), 21–29.

Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536.

Simpson, O. (2013). Supporting students in online open and distance learning. London, UK: RoutledgeFalmer. Retrieved from https://goo.gl/va52aX

Van den Berg, I., Admiraal, W., & Pilot, A. (2006). Designing student peer assessment in higher education: Analysis of written and oral peer feedback. Teaching in Higher Education, 11(2), 135–147.

Webb, M. E. (2005). Affordances of ICT in science learning: implications for an integrated pedagogy. International Journal of Science Education, 27(6), 705–735.

Wiley, D., & Hilton III, J.L. (2018). Defining OER-enabled pedagogy. The International Review of Research in Open and Distributed Learning, 19(4). http://doi.org/10.19173/irrodl.v19i4.3601.