I’ve had a longstanding interest in exploring how students engage critically with automated feedback and develop their feedback literacy. I recently collaborated with the TD School’s Simon Knight and Ajanie Karunanayake, and CIC’s Kirsty Kitto and Simon Buckingham Shum, on a paper that argues why it is so important that we develop these skills in learners.
Let’s get critical
With Generative AI (GenAI) tools becoming widely available, there is a heightened necessity for learners to engage with them critically by understanding its capabilities and limitations. In our paper, we asked the fundamental questions:
- Why do students engage with GenAI for their writing tasks?
- How can they navigate this interaction critically?
Cultivating deeper critical engagement with GenAI is important because it:
- is activity-oriented and targeted at doing learning (so is fundamental to student agency)
- is a metacognitive capacity that both demonstrates and builds student understanding
- is being applied to imperfect models
- requires design for learning
In our approach, we defined in concrete terms and stages how criticality can manifest when students write with GenAI support. We also drew from theory and examples in empirical data (which are still unbelievably scarce in the literature) to understand and expand the notion of critical interaction with AI.
Deep, shallow or absent?
Our framework defines deep versus shallow levels of critical interaction with AI across the following 5 dimensions:
- Planning and ideation
- Information seeking and evaluation
- Writing and presentation
- Personal reflection on AI-assisted learning
- Conversational engagement
These dimensions are then demonstrated through examples of deep, shallow or absent levels of critical interaction observed in authentic written assessment data. Preliminary findings from a group of manually coded submissions revealed that critical interaction was predominately absent or shallow across all dimensions (with the exception of personal reflections).
The results highlight the need for curriculum and assessment changes that go beyond disciplinary skill development. With more insight on how to support this interaction with new AI tools for learning and improving students’ AI literacy, we can prepare our learners to thrive in an AI-driven world.