Much has been made of the impact of Generative AI (GenAI) tools such as ChatGPT on the validity of university assessment and impacts on academic integrity through the use of these tools. While many academics propose embracing GenAI, there is significant agreement that, embraced or not, assessment needs to change to account for the use of GenAI.
GenAI allows students to develop large blocks of cohesive text on almost any topic through a simple prompt. Given this, it is easy to see the allure for students, with the tools cutting down writing time from hours to minutes. This text is more difficult to reliably detect than traditional plagiarism, which is likely to increase the number of students willing to risk its use.
The accuracy of a growing number of detection tools is questionable, particularly with regard to English as an Additional Language (EAL) learners, who are more likely to use digital tools to improve their writing, which can trigger false positives. Similarly, these tools are essentially playing ‘Whack-a-Mole’, pitting AI models against each other. As one improves, it is almost certain the other will improve – and the cycle will repeat, an endless loop chasing accurate detection.
For these reasons, many are advocating for AI-proofing of assessment tasks: ‘designing out’ the risk of AI cheating. This is an admirable, if not always achievable, goal. There are, however, many issues with this approach that are not often discussed. One notable overlooked issue with this approach is the accessibility of assessment tasks for diverse learners.
How do we make assessment equitable?
While we like to think assessment is objective, we know that the design and type of assessment dramatically impacts the ability of students from diverse backgrounds to show their skills and learning. For example, where a majority of students may be assessed in a presentation reasonably well, an Autistic student may struggle to demonstrate their knowledge within the usual neurotypical expectations of this format.
With the rise of GenAI we are seeing a rise in recommendations for assessment changes such as moving assessment into the classroom, limiting written tasks or conducting viva voce interview-presentation styles of assessment. These changes rely on several assumptions which fail to consider the impact on diverse students. For example, in-class assessment assumes students can universally attend and access classes equitably; limiting written tasks assumes that there are not some students for whom writing is the best way they are able to communicate their learning; and vivas rarely consider how much they may increase the stress and anxiety of students already struggling with their mental health.
The use of GenAI in higher education presents as a series of interlinked wicked problems. Notably, how to handle the use of GenAI in classes and assessment, how to minimise its impact on assessment integrity, and how to change assessment to ensure accurate and equitable assessment for all students. While there is no one right answer to these wicked problems, it is our job as academics to tackle them in our subjects the best we can.
Unfortunately, we cannot tackle any of them in isolation. In our reviews of the appropriateness of our assessments in the new GenAI world, we need to be reviewing the equity of our assessments as well. Each assessment should be designed as much as possible to be universally accessible and equitable, and we must consider and plan for how to adjust assessment tasks as needed to accommodate students for which the task proves inaccessible.
What are we assessing our students on?
The main question we need to be asking ourselves, in the context of the current and future challenges posed by GenAI is: what are we assessing our students on? In other words: what knowledge, skills and/or practices are we asking them to demonstrate, and are our assessments accurately reflecting these goals? As you design the assessment tasks for your next subject offering, consider how you can make your tasks more flexible for students and how different diverse students might be impacted by your choices.
The questions below serve as a starting point for this reflection on the accessibility of your assessments:
- Could students choose between modes of delivery when presenting (e.g. having the option of an in-class viva or an online session with just their teacher)?
- Could students choose to complete a written task so long as they provide a version tracked document to show drafting and authorship?
- How would you support the assessment of a Deaf student or students who are blind in your subject?
- How does your presentation rubric account for and provide an equitable set of criteria for autistic students who may struggle with neurotypical body language, eye contact and communication despite having the knowledge being assessed?
- How does your rubric account for EAL Learners: are you assessing their English when the Subject Learning Outcome is about their knowledge?
- How do you know your assessments are universal and are accurately assessing all of these students, even where you are unaware of their conditions and backgrounds?
Resources and support for improving accessibility
Need help with improving accessibility in your subjects and assessments? The LX Lab maintains several useful collections to assist you in understanding and supporting accessibility. The Inclusive Practices team at the LX Lab are also available to speak with subject coordinators, providing technical support and advice around inclusive teaching practice.