For most learners and educators, student evaluations are part of the academic routine: once the subject is done, surveys are released, results are reviewed and changes are implemented as needed. For some, however, the process is not so benign, straightforward or helpful; there are still others whose wellbeing is adversely impacted by student evaluations.
Such issues have increasingly been part of the discussion on student evaluations, from detection of abusive comments in student surveys, to calls for better use of student evaluations. A recent webinar from the Council of Australasian University Leaders in Learning and Teaching (CAULLT) brought many of these conversations together, sharing results from a recent research project conducted with education leaders at regional universities.
The webinar discussions were wide-ranging, seeking to balance the benefits of rigorous evaluation with a more nuanced exploration of how we can better listen to student feedback to improve learning and teaching (spoiler alert: don’t just rely on surveys!). As a researcher in student and academic experience, here’s my own quick take on some key points.
How can student evaluations cause harm?
Student evaluation is a responsibility and regulatory obligation for higher education institutions. Processes are intended to offer opportunities for students to provide feedback on their learning experiences, but should also support educators in reviewing feedback and enhancing teaching.
Some learning and teaching leaders have highlighted how such processes can blur into spaces beyond this scope, becoming a blunt instrument of ‘performance’ measurement, or even a form of surveillance on academic staff. Some studies have found inherent biases in student evaluation reports which can negatively impact teaching staff – in particular, women and other marginalised academic groups. Over the course of a teaching career, this can influence promotional prospects and development opportunities, not to mention the more immediate impacts on mental health and professional confidence.
…but it’s just a survey, isn’t it?
If you ever think to yourself, “Well, a survey isn’t really the right way to make this critical decision, but the CEO really wants to run one. What’s the worst that can happen?”
Brexit.
Erika Hall, Just Enough Research
I love this quote from Erika Hall, and share it often. It may sound light-hearted, but her point is serious: surveys look harmless, but if the planning, design, analysis or presentation of findings is done without care, the consequences can be far-reaching.
When tools like student surveys are administered at scale, we need to be even more cautious. Results are quickly gathered and compared, not only across a faculty or institution, but sometimes between very different institutions with diverse student cohorts, demographics and motivations. What counts as ‘good’ in one institution doesn’t necessarily transfer to another.
From harm to help: multiple sources for positive impact
Whilst we can control for many of these factors and pepper our data with footnotes about why numbers are up, down or about the same this year, that’s not always the point. As speakers articulated in the CAULLT webinar, surveys should not be our only measure, but rather one part of a broader understanding of how well curriculum, teaching and the learning experience come together. Are they working as intended, and if not, why?
The CAULLT Good Practice Guide outlines multiple sources of insight which can be used alongside student evaluation surveys, including teacher portfolios, observation and conversation-driven techniques such as facilitated interviews and group discussions. These may take more time than an automated survey, but educators should be acutely aware of what can be lost in the pursuit of efficient ‘feedback’:
Surveys also shut out paths to genuine learning. Talking to real people and analyzing the results? That sounds hard. Blasting questions out to thousands of people to net a pile of quantifiable data with no gross human contact? Easy!
Erika Hall, Just Enough Research
If we truly value our students as partners, should their experiences really be restricted to the transactional and impersonal interface of the student survey? If research is one of our core competencies in a university, surely we can do better.
Like it or not, student evaluations are here to stay. Beyond mandatory compliance and risk management, they’re part of a broad, connected ecosystem of feedback that helps to inform ongoing improvements in teaching, learning and curriculum. Blend them with other sources of insight, and you might start to shift those surveys from harmful to helpful, and even enlightening.
Further reading and research
Crimmins, G., Casey, S., Weber, I., & Pourfakhimi, S. (2024). Consideration of student evaluations of teaching (SET) and learning: perspectives of learning and teaching leaders through the ethical lens of “first, do no harm.” Assessment & Evaluation in Higher Education, 1–15. https://doi.org/10.1080/02602938.2024.2367586
Crimmins, G., Casey, S., & Weber, I. (2023). Student Evaluation of Learning and Teaching Good Practice Guide, Council of Australasian Leaders of Learning and Teaching
Cunningham, S., Laundon, M., Cathcart, A., Bashar, M. A., & Nayak, R. (2022). First, do no harm: automated detection of abusive comments in student evaluation of teaching surveys. Assessment & Evaluation in Higher Education, 48(3), 377–389. https://doi.org/10.1080/02602938.2022.2081668
Hall, E. (2019). Just Enough Research (2nd ed.). A Book Apart.