Introduction

Students registered in problem-solving courses are typically given a number of assignments and quizzes to complete independently in order to demonstrate and refine their problem-solving skills. It is common for these assignments to be paper-based, and questions are often taken directly from course textbooks. Oftentimes, students rely heavily on their peers as well as on online resources such as solution manuals and websites that allow sharing of course materials and solutions to complete these assignments. This has driven the need to develop a repository of online questions that could be used to create unique assignments with randomly selected questions, each containing randomly generated values.  

Numbas is an open educational resource created by the University of Newcastle, England, as a customizable, online assignment system. Using Numbas, a variety of questions can be created. In this case study, we are creating multiple-choice questions and calculation-based problems for a third year Manufacturing Engineering course. These two styles of questions are intentionally used to assess “knowledge base” and “problem analysis” as two of the twelve documented graduate attributes (set of skills) defined by the Canadian Engineering Accreditation Board.

 Resources

• A computer

Steps for implementation

Step 1. Create a Numbas Account

Go to  https://numbas.mathcentre.ac.uk/  and click on Create an Account near the top right of the screen:

Fill out your credentials in the form and then click on Register.

Step 2. Create a New Project

In Numbas, click on the +New tab near the top right of the screen and select Project.

On the page that appears, enter the name of your project (i.e. the name of the course), as well as a description (optional). You can select the desired default language as well as the license.

You can now begin to create questions that are saved together in this project.

Step 3. Create your Questions

Near the top right of the screen, click on Question in the + Create box. Create all the questions you would like for your first assessment. For help creating questions, view this resource from Numbas .

Step 4. Create your Assessment

After creating all the questions, you are now ready to build the assessment. In the + Create box near the top right of the screen, click on Exam.

Name your assessment (ie. Assignment 1) and select the name of the Project you would like it to be stored in (ie. the name of the project you created in Step 2), then click
Create.

You will then have to select all the questions you would like to add to your assessment. You can choose to use all the questions you have created, or pick a random subset so that each student is given a randomized set of questions (see figure below).

For more information on how to create the assessment, please refer to this resource from Numbas.

Step 5. Share the assessment with your students

If the purpose of the assessment is for practice, and will not be evaluated for grades, you may wish to upload the assessment to the web. However, if you plan to use this as a formal assessment for grades, it is recommended that you upload the assessment to a learning management system, such as Blackboard.

What Can Go Wrong?

Server goes down – If the server goes down, students will no longer be able to access the assignments. In such a case, students who try to access the assignment will see the following error. A technical support staff familiar with operating and maintaining a server is required to resolve the issue. This issue is very rare; however, it happened once during the implementation of this project.

Students do not get their mark immediately after submitting their assignments – Once a student submits their assignment, a pop-up appears letting them know that the exam has finished. However, if the student closes the submission window immediately after this pop-up appears, the grade may not appear in the learning management system gradebook. In this case, when the student attempts to view their grade (in Blackboard, this would be via the My Grades tab), the assignment will be marked as “in progress”.

The instructor can also identify this issue from their view as the submission will be tagged as “incomplete”, despite the fact that there may be a score assigned to it. In the screenshot below, there are two submissions tagged as “incomplete”; one with a score of 0/20, and the other with a score of 11/20. The corresponding students are unable to view their grades in this event. To address this, the instructor must manually enter the grade in the learning management system gradebook.

You notice a mistake in one of the questions while the assessment is already posted – If the assessment is uploaded to the web, any changes will be synced and no other action is required. However, if the assessment is uploaded to a learning management system via a SCORM package, you can edit the desired question(s) in the Numbas editor, download the new SCORM package, and then replace the old SCORM package with the new one. To replace the old SCORM package, open up the assessment in the learning management system, click on Settings, and then select Replace exam package. Any previous and current attempts will remain unaffected in the gradebook column; however, any new attempts will be based on the modified SCORM package.

Further Reading

Numbas documentation (2019). Retrieved from https://docs.numbas.org.uk/en/latest/

Digital resource

To help students learn how to enter their numerical answers and corresponding units in Numbas, we have developed an Entering Answers into Numbas guide

Author

Dr. Sayyed Ali Hosseini is currently an Assistant Professor in the Department of Automotive, Mechanical, and Manufacturing Engineering at Ontario Tech University. He is one of the directors of Machining Research Laboratory (MRL) with research interests focused on Design and Optimization, Manufacturing and Metal Cutting, Modelling and Simulation of Manufacturing Operations, Machining Difficult-to-Cut Materials, Surface Integrity, and Material Behavior. Dr. Hosseini obtained his B.Sc. in Mechanical Engineering with specializations in Solid Mechanics (2005) and M.Sc. degree in Mechanical Engineering with specializations in Applied Design (2008) both from Ferdowsi University, Mashhad, Iran. In 2009, he joined Ontario Tech University and obtained his Ph.D. degree in Mechanical Engineering in 2013. He also held a postdoctoral position at Ontario Tech from 2013 to 2014. Dr. Hosseini is a licensed Professional Engineer in the province of Ontario and a member of American Society of Mechanical Engineers (ASME), Canadian Society of Mechanical Engineers (CSME), and Society of Manufacturing Engineers (SME).

Caroline Ferguson is a Faculty Development Officer at the University of Ontario Institute of Technology. She offers educational support to the teaching community in areas of course development, technological teaching tools, and educational training and support. She is also a Sessional Instructor with the Faculty of Health Sciences and the Faculty of Science, where she teaches undergraduate physics and math-based courses. Caroline’s previous roles included working as a Physics and Engineering Specialist and a senior math and chemistry teacher. She holds a Ph.D. in Materials Science and Engineering from the University of Toronto as well as a Bachelor of Education from the University of Ontario Institute of Technology.