News

This blog was co-written by David Yeats and Ariane Wicks

So you’re worried about how Generative AI could easily answer all your multiple choice quizzes? Here’s PGLD & LX.lab’s top 7 approaches to help you make quizzes more AI-resilient.

Communicate first

Before you start applying the tips, make sure you’ve communicated to your students exactly how they can use GenAI in their subject and familiarise them with UTS’s new Academic integrity homepage. Both of these can be downloaded into your subject through Canvas Commons.

The question of quizzes

Wherever possible, avoid online non-invigilated, self marking, multiple choice quizzes (MCQ’s) in Canvas for large components of student assessment (over 20%). But sometimes you need to use them, so the question is: Can you create MCQ’s that are more resilient to GenAI? The answer: Yes, indeed! (with provisos) 

Currently AI still has limitations in understanding context, reasoning and thinking critically the way humans do.  You’ll need to use these strategies in combination to be more effective, like a Swiss Cheese approach. Using only one approach is not going to work. These strategies are intended to put up more challenges than usual in online quizzes, however, don’t think of them as a silver-bullet. There are many work-arounds if you’re determined enough.

7 quiz question tips

The following are some strategies to consider, with Before and After examples.

1. Avoid simple factual questions

Questions that require only factual recall can be easily answered by AI systems. Consider questions that require higher-order thinking skills like analysis, synthesis, and evaluation.

Before: Canvas MCQ which asks ‘What are the 4 indicators of lifelong health covered during the Personal Health Record assessment?’. There are 3 answer options. After: Canvas matching question type which provides a Health Worker scenario about a baby’s growth and development and the 4 indicators of lifelong health and parental wellbeing. There are 2 further detailed scenario statements which need to be matched to 1 of 4 indicators in a drop-down menu.

2. Use highly specific or local information

By referring to highly specific knowledge such as small local events, in class discussions, lectures or other course materials, it’s harder for AI to rely solely on pre-existing information.

Before: Canvas MCQ which asks ‘What were some of the strategies for managing comorbidities in senior patients’. There are 4 answer options. After: Canvas MCQ which asks ‘In Module 1 we met Mrs Chang and learnt about her health history and current situation. What were some of the strategies used to manage her comorbidities?’ There are 4 options.

3. Consider ambiguity and reasoning

Reword your questions so they are more ambiguous or offer multiple correct interpretations. AI does not deal with ambiguity or complex reasoning as well as humans do.

Before: Canvas MCQ setup for True/False which states ‘The complexity of ecological risk assessments is greater than that of human risk assessments.’ After: Canvas Multiple Answer question type that asks ‘Which of the following are reasons for ecological risk assessment being more complex than human health risk assessment?’ There are 4 options.

4. Use distractors strategically

Distractors (the incorrect options in multiple-choice questions) should be plausible and align with common misconceptions, challenging AI with critical thinking skills.

Before: Canvas MCQ which asks ‘What is the duration of copyright protection for works created by an individual under the current copyright law?’. There are 4 numerical options. After: Canvas MCQ which asks ‘Which of the following statements about copyright duration is incorrect?’. There are 4 statement options.

5. Get creative with language

Consider using metaphors, analogies, idiomatic expressions, humour or even sarcasm. AI may not understand the tone or context to answer accurately. 

Before: Canvas MCQ which asks ‘Energy is…’. There are 4 options. After: Canvas MCQ which asks ‘Which of the following analogies best represents the concept of energy?’. There are 4 analogies as options.

6. Experiment with different types of questions 

Vary the question format used to include ordering and sequencing tasks, matching or multiple dropdowns to increase the complexity of your quiz. Copying and pasting the question from Canvas into an AI tool will take as long as simply working out the answer.

Before: MCQ that can be very easily answered by copy and paste. Alt: Screenshot of a Canvas MCQ which asks ‘What does SoLAR stand for?’. There are 5 options. After: Canvas MCQ which asks ‘What does SoLAR stand for?’. There are 5 options.

7. Use multimedia

Incorporate images, audio, or video in the question prompts, or the possible answers to make them more contextually rich and challenging for AI to analyse.

Before: Canvas MCQ which asks ‘Which of the following amino acids used for protein synthesis are not synthesised by humans?’. There are 4 options. After: Canvas MCQ which asks ‘Which of the following amino acids used for protein synthesis are not synthesised by humans?’. There are 4 molecular structure image options.

Other key strategies

Don’t forget other key strategies to enhance the academic integrity of your assessed quizzes  and prevent cheating such as:

Still got questions? The answer might be in Assessment, academic integrity and GenAI at UTS – your Town Hall questions answered – LX at UTS. If not, do yourself a favour and get over to the events currently running at the AI x L&T week. 

Join the discussion