Here are some tips for designing a good open book exam.
Make it about what students do
The key to open book exams is to set questions where students need to do things with the information they have at hand, rather than simply locating it in the text or notes during the exam and reproducing it. Try to make things students do in the exam mostly in the ‘higher order’ levels of Bloom’s taxonomy (see below). Things like creating, evaluating, analysing and applying knowledge – with marking criteria in place to suit of course.
University of Newcastle’s handy guide to open book exams recommends starting by presenting some pertinent qualitative or quantitative data, and then asking questions that involve interpretation or application: ‘What does the data show? What relevance does this data or does the scenario have in terms of [component of current topic]? What other factors could potentially affect this data? How would you test for these?’ Here’s an open-to-closed question makeover example from an Engineering course at Hong Kong Polytechnic University (on page 4).
Open book exams can lend themselves to questions that are out of reach of most closed-book exams. Multi-step questions, where one question builds on the previous one can work well. Questions that are broader in scope and require students draw together content from right across the curriculum, are also more feasible than in traditional exams. See ‘Open book examinations for assessing higher cognitive abilities’ for a detailed explanation.
Need entry level questions as well? Your easier or shorter questions could ask students to do things like classify or compare pieces of information from their support materials, rather than just rely on pure knowledge reproduction (see Mohan Dhall’s article, Fear Not the Open Book).
Real world curriculum, real world questions
In designing assessment for your subject, think about the curriculum areas that are best suited to an open book exam. For instance, topic areas which ask students to put themselves in the shoes of a professional in their field (for example, ethical considerations when working as part of a professional team) can easily translate to an open book question, as to replicate real-world conditions students would probably draw on other material, such as industry regulations or professional standards. Here’s an example from physiotherapy at the Manchester Metropolitan University.
Business academic Jeremy Williams suggests that when framing a question, setting students up as the ‘expert advisor’ or key decision maker for a realistic question that they might encounter in professional practice, can be an effective way of connecting them to the point of what they are learning. Williams also suggests keeping an eye out for ideas for good, current ‘real world’ open book exam topics in the real world – i.e. the popular media. When dealing with a real world scenario, Williams reminds the students of the learning outcomes, so that they have the ‘maximum opportunity to demonstrate that they have achieved these learning outcomes’ – also stopping them, I suppose, from getting distracted by irrelevant details of a case study.
Open-web exams?
You might want to consider whether an open-resource or open-web exam, where students have access to web-based resources, might suit your subject. In health professions for example, student use of online or personal resources in exams could result in a shift in the focus of learning away from libraries and toward ‘rich, but messy, clinical learning environments’ they will be working in post-graduation. All the same principles for designing questions in an open book exam apply: how can you get students to show you what they can actually do, granted the fact they have Google at their fingertips?
If you’re running an open book exam using an online quiz, it’s a good idea to keep using anti-plagiarism techniques like randomising question order, using question pools and calculated formula questions. These methods reduce the payoff of cheating – regardless of whether your quiz is open book. (Note: there are many other considerations about running and invigilating online exams, which I’m not going to wade into here.)
And 3 practical tips for managing open book exams
1. If you allow student-generated materials (notes, crib-sheets, flashcards etc.) in the exam, consider devoting some class time to preparing these, or build it into a reflective learning activity in your subject. Students could also work on producing these together and discuss ways they might apply the content to different scenarios.
2. Most sources I consulted in writing this post suggest that time management is vital to a successful open book exam. Open book questions typically take longer to answer, so make sure the time allocated is realistic, even by testing it out yourself. Minimise the time students will waste working out what they need to do by wording your questions very clearly.
3. Almost goes without saying, but make sure you remind everybody that your exam is open book. Especially for new students, and while students are getting used to the new UTS assessment policy, this could be unfamiliar territory. Be clear to students on the guidelines for the exam, what materials they can bring in and what is expected of them.
Disclaimer: I’ve never had to design an open book exam, and the last one I sat (not at UTS) broke just about every one of these rules. If you’ve got some tips based on your real world experiences – good or bad – please share them in the comments!
[…] How to design a good open book exam and decision […]