York University

Project Description

Labster virtual labs were incorporated into BIOL1500, a face-to-face introductory biology bridging course that serves as a “12U” (senior high school course) replacement. Students completed the virtual simulations on their own outside of class time. They then used the knowledge and experienced gained in the simulations to complete in-class activities. The simulations and associated in-class activities helped students reach core course learning objectives.

Originally the goal for this project was to use Labster virtual simulations as a complement to in-class content. Virtual simulations would allow for the visualization and review of difficult concepts and further enable self-directed learning. As BIOL1500 does not include a wet-lab component, Labster would also allow students to experience the process of scientific investigation in a hands-on fashion, adding an experiential element to the course structure.

We believe these goals have been reached and the project has been successful. Students expressed satisfaction with the virtual simulations; many of them self-reported revisiting the simulations to study for tests and better understand material. After completing the simulations students came to class asking more probative questions, having deeper conversations with peers, and being more engaged with the related in-class material.

Students said that simulations were “fun” and “cool” several times, and there were no complaints about the simulations themselves. There were a few technical issues, which are addressed below, but overall students enjoyed the experience.

One additional goal of the project was that the use of virtual simulations allowed students to gain core lab competencies that would help them succeed in future life science coursework and professional opportunities. Students have not yet moved on to future life science coursework or professional opportunities, so it remains to be seen if this goal was reached. However, the simulations prompted several discussions about “real” labs in class and career options in biology. These discussions suggest students were treating the simulations as “real” lab experiences.

The vision and goals of the project did not change over the process. However, major structural changes to the course were required due to the labour disruption at York. BIOL1500 was suspended throughout the strike (starting March 5, week 7 of 12). The course resumed at the start of the remediation period (July 22), but was completed in a very individualized fashion and no longer face-to-face. This disruption and the necessitated course changes make outcome comparisons to previous course offerings without virtual simulations difficult (post-week 7). The disruption also changed the plans for project analysis. That being said, students visited the virtual labs during the strike and used them to stay connected to course material even while the class was suspended.

Team Description

There were numerous discussions with, and support from, the executive/management team of the Biology department (undergraduate program director, chair), Faculty of Science (associate dean, students) and York University (associate vice president, Teaching and Learning) as well as the Scholarship of Teaching and Learning team at York’s Teaching Commons.

The closest and most frequent communication was with Alex Mills (associate dean, Students, Faculty of Science). Discussions were often held on the project goals and status, and he assisted in updating other stakeholders, mainly the AVP Teaching and Learning. The staff at the Teaching Commons reviewed the course plan and the research ethics application before the start of the semester, providing expert guidance and feedback. Discussions with the chair and undergraduate program director of the Biology Department also happened at several points throughout the year.

We also worked with the Labster implementation team to ensure smooth integration of the virtual simulations into the course structure and to keep them updated of any issues arising. Representatives of Labster and eCampusOntario were contacted during the labour disruption to alert them to the changing needs of the course, and they were able to assure Labster licence extensions to support students throughout the course remediation process.

We created material to help learners connect virtual simulations to course learning goals, and integrated Labster with in-class experiences. Students were provided with multiple online and in-person means through which they could ask questions and learn more about the virtual simulations project.

Integration Details

BIOL1500 started with an enrolment of 120 students (January 8; 100 as of July 22). Virtual labs were integrated into the course and were a component of the final grade. Therefore, all students should have registered and used the virtual simulations (I discuss usage rates below).

Students were assigned five mandatory virtual simulations with an additional four optional simulations. For the mandatory virtual simulations, participation marks were assigned for finishing the simulation by a certain date. For four of the five mandatory simulations there were also associated in-class activities where students worked in groups to learn more about the topic and their learning was assessed. Of those four in-class activities, three were completed before the strike (March 5). The labs were aligned to key learning outcomes of the course, and completing the simulations along with the in-class activities helped prepare students for tests and exams.

Students were provided with a virtual labs assignment outline and individual instructions for each lab at the start of the term. While students had to complete labs by a certain due date to receive participation marks, they could complete the labs at any time. The instructions for each lab briefly describe the lab content, connected the simulation to the course learning objectives, and provided a completion checklist. The checklist helped situate the simulation into the course by providing textbook and other readings to help familiarize students with background material that would help them understand the lab, described the in-class activity, and connected the simulation with what we would be learning in the future.

Only one educator used these virtual simulations at York University. That educator was teaching five other courses concurrent with BIOL1500 and had a secondary position supervising student leadership programs at one of York’s colleges. Support was available at the departmental, faculty, and institutional levels, but all decisions about course structure and addressing day-to-day learner needs was the educator’s sole responsibility. Given this, the excellent support provided by Labster and the ease of use of the virtual simulations were strong benefits to the project.

The speed at which Labster responded to issues and the number of times they checked in to ensure things were running smoothly was impressive. The labs were also user friendly. There were very few usage questions from students; they seemed able to navigate the system and work out any issues on their own.

There can be conflict between what improves student learning outcomes (mainly active learning) and how students believe they learn best (lecture). Some students feel they are being forced to “learn on their own” if they are not lectured on a topic. There were no complaints about Labster. Students seemed to genuinely enjoy the experience. Witnessing the animated discussions and debates as they worked through material in the associated in-class activities was very rewarding. It was clear students were engaged and were thinking deeply about the subjects.

The main challenge was the cost of the simulations and the technology requirements. There are on-campus computer labs that are capable of running the simulations, and information on these labs was provided to students. However, we are a commuter school where many of our students work and/or have family obligations outside of class. This makes computer lab use less convenient, and it is possible that participation in the labs would decrease  if students don’t have the computers necessary to run the simulations at home. That being said, only one student mentioned this concern, so other than needing to be prepared with on-campus solutions, it was not a large challenge. The cost of the simulations could be a future challenge, but one of the biggest benefits of this project was being able to provide licences to students for free in this pilot offering. It allowed for focus on integration and other issues, ensuring there was a benefit to the simulations, before burdening students with the cost.

User Experience

Simulation completion dates were tied to when the associated learning objectives would be covered in class. Most of the virtual simulations corresponded to earlier course material, and the first four mandatory labs were completed between January 11 and February 6. Labs were scheduled so that there was never more than one to be completed in a week. The schedule of labs and due dates was prominently posted on the course website and all dates were listed in the assignment outlines. Students still had many questions about scheduling and timing at the start of the semester. These questions decreased after the first lab was completed, and for the most part students were able to follow the due dates without issue.

There were 118 students enrolled in Labster (i.e., registered their licence; class size approximately 120). The number of students who completed the virtual simulations in the pre-strike mandatory labs ranged from 86 to 110, a good participation level. In comparison, of the 120 students enroled at the time, only 105 wrote the in-class mid-term. Three of the four labs saw completion rates of 93 to110 students. The mandatory lab that the least number of students completed (86) was the simulation not associated with an in-class activity.

The lower completion rate of the mandatory lab not followed by an in-class activity (86 students) and the extremely low participation in non-mandatory labs (where subjects were covered before the strike; four to 11 students) suggests that integration of virtual simulations into in-class components, and into grading scheme, is necessary for good student buy-in.

Interestingly one of the mandatory labs (Evolution) had a due date for completion of March 22. The labour disruption started on March 5 and when classes resumed (July 22) the lab was made optional. Students could complete the simulation if they wished and have it count toward their grade, but if they did not complete it they would not be penalized. Nineteen students completed the simulation after the start of the strike. This suggests, at least for some students, the virtual simulations were key self-study tools.

Labster provided excellent support and the virtual simulations were very user friendly. Labster provided handouts for students on how to register for their site, technical requirements, and support options. These handouts were very useful and helped address many student questions and concerns.

While students were provided with information for the Labster support, we also set up a “trouble” form on Moodle (our course LMS) that could be submitted anonymously, and encouraged students to send an email if their issues could not be resolved by Labster. Students felt no need to report issues anonymously, and the handful of technical issues that occurred were either signed (on Moodle form) or through email. These issues were small in number and minor. Most were about inability to access their account, which was mostly a user error that was quickly addressed once the concern was forwarded to Labster.

Value

Data gathering and analysis were the aspects of this project most heavily affected by the labour disruption and suspension of BIOL1500. We intend share experiences with all stakeholders including at departmental, faculty, and institutional meetings.

The virtual simulations were incorporated into the course structure so that much of the data needed would be generated as part of mandatory coursework. In addition to this coursework data, a student survey was designed to address learner experience, which was to be delivered to students after final grades had been submitted.

York’s Senate policy on research involving human participants guided the assessment planning, and work was done with York’s Teaching Commons to finalize assessment design and submit a proposal to the research ethics board. The Teaching Commons advised that delivering informed consent documents was best left to closer to the end of term as the only data not part of mandatory coursework (the experience survey) would not be delivered until after the course ended. Unfortunately at the start of the labour disruption (March 5), we were still awaiting final ethics approval, and informed consent was never received from any student.

Despite the issues mentioned above, an evaluative approach to assessment can be taken using the coursework-generated data that respects the rights and confidentiality of students. However, the strike still necessitates some changes and has led to a delay in when coursework data will be available.

Students seemed to thoroughly enjoy using the labs. After completing a virtual lab many of the discussions and questions from students about associated content were at a deeper level than usual. We believe it was a positive learning experience for my students.

The original goal was to design pre- and post-simulation assessments to measure student learning gains after using Labster. Questions were used from a standardized diagnostic for introductory biology as part of a learning activity during the first class, which provided a pre-simulation assessment of learner knowledge (Shi J, et al., 2010). The plan was to incorporate the same questions into the final exam allowing for pre- and post-simulation assessment. While this assessment would capture all learning in the course, not just simulations, it would provide a standardized way to track learning gains if carried across years (e.g., if in future offerings the type and number of labs change). The diagnostic questions used were also specific to the learning objectives covered by the simulation. Unfortunately this planned assessment is no longer possible as post-strike remediation necessitated changing the final exam format to a take-home written test; also, students no longer have to complete the final exam.

Student responses to, and grades on, the lab associated in-class activities can also be used to assess student learning gains from the use of the virtual simulations. Three of the four planned activities were completed before the strike.

The course structure originally included two mid-terms and a cumulative final exam. The plan was to add bonus questions to each test that covered information the students would have encountered in the labs. The questions would be related to a course learning objective but a bit beyond the level we’d cover the material in class. The intent was to assess retention of information from the simulations themselves instead of other class material. Only the first midterm was held before the strike, and on that test there were three of these bonus questions. Between 54% and 68% of students correctly answered the questions (depending on the question), and only 10% of students answered all questions wrong. This suggests students were learning and retaining information from completing the simulation alone.

Many of my final exam questions were designed to serve an evaluative purpose. They were mapped to learning objectives, and correlated well with student learning. If students had completed the original final exam during this project comparing their performance on these key questions with previous, non-simulation semesters, it could have provided interesting information. This may be possible if virtual simulations are used in the future.

Future Plans

The use of virtual simulations in BIOL1500 was a success. The use of virtual labs helped students to more actively engage in course material and reach the course learning objectives. Despite the disruption and course changes that make a full analysis of project results difficult, it would be helpful to use Labster or other virtual lab simulations again.

It was always the intention to continue integrating virtual simulations into BIOL1500 if this project was successful. The experiences were positive enough  very early on in the semester to decide that integration should continue into the 2018–19 school year. Unfortunately, with the changes necessitated by the labour disruption, the continued integration of Labster for the fall 2018 was not possible.

One major concern for quality and sustainability is the cost of student licences for Labster. York has an ancillary material policy that caps online tools used for student assessment at $60 (like textbook associated quiz platforms). While the ancillary policy may not apply to virtual lab simulations, the spirit of the policy—keeping courses financially accessible—should be honoured.

A handful of students were able to discover the approximate $80 licence fee for Labster either by exploring the site on their own or actively searching for the information. They reported that they were glad to have free access since they would not have been able to afford the licence. This further affirms that the cost of Labster access would be a hindrance to future use.

It was the instructor’s practice to always waive participation for those who could not afford it when ancillary material was used in a course (usually textbook-associated quizzing platforms). If necessary, alternative assessments were offered. The integration of Labster into BIOL1500, which worked quite well, involved in-class activities completed in groups. These activities allowed students to reinforce the concepts they were learning in the simulations, and to use the labs to build group work skills which enhanced in-class discussions and other active learning experiences. If costs prevented some students from accessing Labster, the quality of the learning experience for all could diminish.

One way to mitigate the cost aspect is to switch BIOL1500 to an open textbook. Currently the mandatory course textbook is approximately $120. While some students buy a used textbook or the e-text (approximately $100) most students buy a new textbook. If instead the textbook were fully open and free, the incorporation of Labster would not be as large of a burden to students. There are many sources of open textbooks for introductory biology, but they would have to be evaluated to find the right fit. This will not completely solve the cost issue; some students will still be unable to afford Labster. With the support at the departmental and faculty levels to help come up with additional solutions, and if an appropriate open textbook can be found, future use of Labster in BIOL1500 is likely. There may also be lower-cost alternatives to Labster that could be used in a similar manner. That being said, given the positive experiences during this project, we would be reluctant to go this route.

We had discussions with Alex Mills (associate dean, Students, Faculty of Science) about using the Labster virtual simulations as the starting point for flipping and blending BIOL1500 extensively. The way Labster was integrated for this project lends itself easily to a flipped and blended structure where students would complete labs on their own time, focusing class time on associated activities. Other course material would be covered in a similar fashion with online readings, lectures, or other components being completed before class and the students using class time for more active learning.

With the support of Alex Mills, the plan was for the instructor to spend the summer both analyzing this project and preparing BIOL1500 for the above-noted structural change including the continued incorporation of Labster virtual simulations. In fall 2018 the hope was to run the course as a flipped learner experience with Labster virtual simulations and using an open textbook. Then by winter 2019 BIOL1500 would be restructured into a blended course offering (defined as at least one-third of class offerings being replaced by online/other components). This was not possible because of the consequences of the strike.

We still believe the plan outlined above is possible, sustainable, and would improve the quality of the learner experience. It is therefore still the goal to incorporate Labster in future versions of BIOL1500.

Lessons Learned

References

Shi, W. B. Wood, J. M. Martin, N. A. Guild, Q. Vicens, & J. K. Knight. (2010.) A diagnostic assessment for introductory molecular and cell biology. CBE-Life Sciences Education, 2010, 9, 453–461.