Assessment Security and AI: Part 1

• academic integrity
• assessment design
• assessment security
• authentic assessment
• contract cheating

Earlier this year I attended a webinar hosted by the excellent Transforming Assessment program and presented by Phillip Dawson on the topic of Detecting and addressing contract cheating in online assessment.

Phill is a leading researcher at Deakin University in assessment, feedback and cheating in higher education. He is also the Associate Director of CRADLE, the Centre for Research in Assessment and Digital Learning. The following post is a recount of the key ideas and issues Phill has presented. Almost all the research cited is his and the ideas are taken directly from his work.

Assessment security online

There’s been a lot more interest in contract cheating since so much of teaching and learning has moved online. According to a 2020 survey conducted by Wiley, 93% of teachers thought that students were more likely to cheat when being assessed online.

While we’re yet to see evidence of contract cheating being more prevalent in online assessment, it’s also true that our ability to detect contract cheating has been extremely limited until very recently.

First things first, there are a few statements we should make about this topic before getting into the core content:

This series is in three parts, of which this is the first.

1. Contract Cheating is a giant problem
2. We need both Academic Integrity (positive preventative mission) and Assessment security (robust processes in place to ensure assessment occurs under the conditions educators lay out)
3. We can learn to detect contract cheating

Now, apologies if you’ve read this far and wondering when we’re going to write something about AI – Artificial Intelligence. This blog is really only dealing with AI – Academic Integrity. Artificial intelligence does play a role in Assessment Security, but not yet a big one.

CC is a big blurry problem

There are three key things that make contract cheating (CC) a real big problem: Students don’t just find contract cheating, it finds them. A significant proportion of students contract cheat, but contract cheating is not limited to students. Only a tiny fraction of contract cheating gets discovered.

How companies target students

• Usually we think of buying essays (eg. someone goes to a search engine and they type in ‘how to buy an essay’) – this does of course happen, but there’s a lot more to it.
• Even fairly benign searches can make you prey to contract cheating companies.
• Type ‘Essay writing help’ into a search engine; go on to social media and type in ‘research paper’; ‘due tomorrow’; ‘hate stats’ (Amigud, 2020) and you will likely be bombarded by so-called tutoring or homework help companies.
• Edubirdie, one of the most well known contract cheating companies gets promoted in banal magazine articles or gets advertised by sponsored YouTubers.
• Guerrilla marketing by such companies often takes the form of on-campus parties.

One thing we know is that these services use sophisticated and manipulative ways to get to students and hold them.

What’s the prevalence?

Well, many readers will know of the groundbreaking study by Tracey Bretag et al in 2018 for Australian higher education which found 6% of students engage in behaviour that can be considered contract cheating. Extrapolate that to 80,000 nationwide, and it becomes an average of 2000 cases per uni. Given that this was self-reported, we can assume the actual number to be higher.

UK universities have not been subject to the same kind of study that Australians have, but it is clear that the cases they reported catching are far fewer than the cases claimed to be occurring (Newton, 2018). Precisely, less than 1% of cases ever get caught.

So what to do?

Banning essays won’t work. Authentic Assessment won’t work. Contract Cheating can occur with any task type (Ellis et al, 2019 ‘Does authentic assessment assure academic integrity‘). We can and should do assessment better!!! But it simply will not stop CC.

Rapid turnaround times won’t work. Contract cheating services can do it quicker than students! (Wallace and Newton, 2014, Turnaround time and market capacity in contract cheating). There are so many people waiting to provide work for students, it’s not like students are waiting in a queue to have their turn – people will jump on them as soon as they need it and the turnaround is only dependent on how much they can pay. Anything can be expedited for the right amount.

Exams won’t work. Third party cheating is more common in exams than assignments. It’s also less likely to be detected in exams than assignments. (Harper, Bretag & Rundle, 2020 Detecting contract cheating: examining the role of assessment type)

Remote proctoring might help: but the vendors won’t let researchers test it out… yet.

Phill Dawson is attempting to conduct research into online proctoring software. He would like to do a study paying professional cheaters to attempt to cheat on those platforms. He has approached several remote proctoring companies and received funding for the research, but they have refused him access to test their software. They state that it’s against their terms of service. So their tech might work, but so far no-one has been able to rigorously test it. Treat the claims of e-proctoring services with caution.

That said, we in the LX.lab know that if you’re running an online exam, you need to verify identities of exam takers. One of the few ways to do this with large classes is by using AI-proctoring software. You can find more information in our LX Resources on Online Exams.

If you’re interested in this particular area, Phil writes much more about this in his book: Defending Assessment Security in A Digital World now available in the UTS library.

Tough questions

So far we’ve looked at what students are doing but what about teaching staff?

What do we do when academics have very tight turnarounds?

Indeed, the time pressure on academic staff is increasing not decreasing. This is a major threat to the integrity and security of assessments, not to mention the integrity and security of academia more generally. That said, while the approaches to assessing the integrity of student work don’t take extra time on the detection side of things, proving a case of contract cheating is incredibly time intensive, as anyone who has attempted it will know.

So even if you think you’ve found a case of contract cheating, what do you do? That’s not a question we can answer here yet.

Is designing open book tasks the way forward?

Anytime we have restrictions on students, we need to be able to enforce those restrictions. Closed book exams have restrictions that open book exams don’t require. They give us more restrictions that we need to enforce. If we can’t enforce them, then we’re making students targets for contract cheating.

So open book exams ironically give fewer opportunities for contract cheating and collusion to occur. An instance of contract cheating is far easier to detect in an open book exam because in all likelihood, the responses will not use any of the open-book material – but that will be dealt with in part 3 of this blog series on Assessment Security and AI.

That’s the end of part 1. Keep an eye out for the next posts in the series, where we’ll explore these issues further.

Feature image by Kenaz Nepomuceno from Pexels

Share

Twitter Facebook