Some simple things to make my Online Exam better

Be transparent
Explain how questions are marked (e.g. multiple choice questions: 0 or full points, partial points*, negative points**?)

* LET has developed some new question types (see https://blogs.ethz.ch/letblog/2019/12/18/how-to-multiple-choice-with-moodle/) that allow you to choose how you want to award points. We offer recommendations on how to use them and are very happy to chat with you about the pros and cons.

** LET strongly argues against the use of negative points, we can gladly explain to you why we think that way. If you are interested, ask us!

Check thoroughly that what you are telling the students about the scoring method actually corresponds to what is implemented in Moodle. If you are unsure how to check this, ask us!

Mix it up
Combine closed and open question formats. Closed questions are also called “selected choice questions”. These questions are very time efficient and may be scored automatically. They are good for assessing students’ breadth of factual knowledge.

Open questions are also called “constructed response questions”. In these questions, students have to generate answers themselves. These questions usually require manual grading and are good for assessing depth of knowledge.

Important: Be transparent about this well before the exam. For the student, it makes a huge difference in their preparation for the exam if they know there will also be open questions. A savvy student knows to prepare quite differently to an exam that also contains closed choose-your-answer-from-the-list-questions.

Be consistent
Especially if several lecturers contribute questions to your exam, you have to make sure that the same scoring method is used within the same question format.

Also, check whether you can simplify the exam by using only a select few question types. For example, do not use “Kprime(ETH)” when most of the other questions used “MTF(ETH)” for this type of question (“MTF” stands for “Multiple True/False”). For more details, see this blog.

Structure your exam
You can use additional description fields or section titles in your exam to show the students that questions from the next topic or examiner are coming up.

Maybe use a description field at the very beginning of your exam that answers the students’ most pressing questions, e.g. how many questions there are, how many points can be achieved, how the questions are scored, how much time they have, whether (or until when exactly) they can hand in their exam early and so on. You lessen the mental load that is not relevant for passing the exam for your students when you write down this kind of information. Be aware that there are different description fields available in Moodle that are only visible before the start of an exam or only visible after the exam has started.

We strongly recommend showing only one question per page. This makes the exam clearer. In addition, this is the only way to make sure that students’ answers are always saved promptly since Moodle saves their answers when they click on “next page”.

Get feedback
Especially if you have decided to only use choose-your-answer-from-the-list-questions, add a final question at the end of your exam of the type “Essay” with 0 as the default mark. Ask students to use this question to give feedback regarding specific questions that were not clear to them. You can use this feedback to improve your future exams. Most importantly, it will give students the possibility to voice their concerns and make them feel heard and respected.

It should be obvious that – as with all other parts – you have to be transparent about what is going to happen with the students’ comments. If you tell your students that their comments will be read, then please actually make sure you have the resources to do that.

Provide enough time
As before, ask yourself: what are your learning objectives? Is a high speed of answering a learning objective? No? Then don’t test it. Yes, very fit students will usually be faster than less fit ones. But is this important for their later achievements? Does that mean they are better engineers? Will they ever have to “vomit facts” (a student term for this) as fast as possible in their future job?

If you are interested in assessing your students’ depth of understanding, we do not recommend adding time pressure. An exam is highly stressful situation in itself and there is no need to exacerbate this stress even more. If assessing depth of understanding is your concern, additional stress usually will not help distinguish between the skilled and inept students.

Test for skills
Make your exam as authentic and resource-rich as you can. Think about what you expect students to do with what they have learnt in your course, rather than what you expect them to know. For example, do an exam with third-party applications (like Excel, R, Matlab) or an open-book exam where students can have their own notes, lecture notes or many other documents available. Especially if your course also consists of exercises where the students use specific software, contact us about the possibilities to make your exam as well-aligned with your course (and your learning objectives) as possible. If you have other ideas about how to make your exam more authentic, contact us as well please.

Be prepared
Attend the workshop invigilating online exams. Do it. At least once in your life as an Online Examiner at ETH you must have attended this workshop. And of course, you are welcome to join us again if you want to refresh your knowledge or get updated with our most recent tips and tricks and experiences.

If you don’t know when the next workshop takes place, ask us! Make sure that in each exam room you have enough supervisors that have attended the workshop. Imagine there are 230 students who have to wait because two students cannot login because they have accidentally hit Caps Lock at some point because they are so terribly nervous – and you are not there to help them. This is a bad situation. We know it’s not fun for the 230 individuals who have to wait in front of their (maybe) dreaded exam. And it’s for sure not fun for these 2 students keeping everybody waiting. Put yourself in your students’ situations.

Our general rule of thumb for the required number of supervisors is this: In any room you need at least 2 supervisors, because – toilets! Then, for the larger rooms (e.g. ONA E7 and HG G1), if you have more than 50 students in your exam, add another supervisor. For 50 more, add another. For example: for 75 students you need at least 3 supervisors, for 101 students you need 4 supervisors, for 240 students you need 6 supervisors and so on.

Inform well and early
In many exams, we provide you with stickers that have the students’ names. You distribute these stickers in the rooms (one for each computer, usually in alphabetical order) such that each student has one specific computer allocated to him or her.

Make it as easy and relaxed as possible for the students to find their computers. Inform them well where they have to be. In large rooms, label the rows clearly (see photo below). Maybe even hang up posters in the waiting areas showing them in broad terms where their computer will be so they can make a beeline for it and don’t have to crouch down to read the small print on every little sticker.

From the very beginning, be very clear what they may bring to their workstations and what they may not (maybe even in an email before the exam). Imagine an exam where most of the students do not have to go back and forth between their workstations and the place where they have put their stuff. Heaven!

Make sure that students in different rooms or even different locations (Oerlikon and the Main Building) receive exactly the same information before and during the exam. Set up a communication channel with your team prior to the exam and send relevant information (questions asked, mistakes found, …) immediately to all other rooms involved in your exam.

Think about the end
Well in advance, think about how you want to end your exam. Do you want to allow the students to leave at any point? Only at the end? Be aware that the exam might end for some students 3 minutes later – either because they have started later for some reason (e.g. inserting ear plugs) or they encountered a technical hickup (e.g. their mouse stopped working) and were allowed 3 min additional time. Now, if 227 students finish within the same minute and they all get up (can you hear the chairs screeching on the floor? <insert sound file here>), this means a lot of noise. They usually start talking and comparing their experience immediately, which is completely understandable but very hard for the student who cannot make use of his or her additional 3 min. Ask your colleagues how they do it. Ask us what different styles we have witnessed.

Respect our guidelines and deadlines
We thought about this. We actually have. Please use the checklists we provide. Please send us the things we need from you at the very latest at the deadlines we wrote you by email. An exam situation might turn sour quickly if we did not have enough time to test or prepare it. Imagine again your 230 students sitting in front of the computer and not one student can enter the exam – then you realise you have forgotten to check the settings. How fast are you in finding out where the mistake was and fix it? Or imagine some (but not all) of your students being able to see their achieved points at the end of the exam – how will the different students react to this? The checklists we provide help you in making sure these kinds of scenarios do not happen. If our checklists do not answer your questions, ask us!