It’s fascinating how we try to pin down learning, isn't it? Especially in education, where the goal is to measure something as fluid and personal as a student's understanding. Recently, a report delved into the views of Assessment Organisations (AOs) regarding potential pitfalls in how we assess qualifications, particularly within the CASLO approach. It’s not about pointing fingers, but more about a collective deep dive into what could go wrong and, crucially, what’s being done about it.
When these AOs were interviewed, they were asked about a list of potential assessment problems that have been discussed in academic circles. The interesting part? Most of them recognised that these issues could be relevant to their qualifications, even if they weren't necessarily current problems. It’s like acknowledging that a certain type of weather could happen, so you prepare for it.
So, what were these potential problems? The one that popped up most frequently was the idea of inaccurate judgements. This is where assessors might misjudge whether a student has met the required standard, leading to some passing when they shouldn't, or vice versa. The report highlights that this can stem from assessment criteria (AC) being tricky to write and interpret precisely. When AC alone can't clearly define the line between passing and failing, or between different grades, it puts a lot of pressure on assessors who have to rely heavily on them.
Another area that came up was atomistic assessor judgements. This is a bit more specific, suggesting that perhaps assessors are focusing too much on tiny, isolated pieces of evidence rather than the whole picture. Interestingly, this was the least recognised problem, though some AOs did see a degree of relevance in it.
Then there's the concern about poorly conceived assessment tasks or events. If the task itself isn't well-designed, it’s naturally going to be harder to assess fairly and accurately. Following that, the report touches on lenience and malpractice, which, as you can imagine, are serious concerns in any assessment system. And finally, inappropriate support – the idea that students might be getting too much help, which then skews the assessment of their own abilities.
What’s really encouraging is that the AOs didn't just identify potential issues; they also discussed the mitigations they put in place. These are the practical steps, the protective factors, that help to either prevent these problems from arising or lessen their impact. The report notes that whether an AO recognised a problem or not didn't significantly change the types of mitigations they discussed. This suggests a proactive approach across the board, a shared understanding of the need for robust assessment practices.
The report goes on to analyse these AO views, looking at the relevance of these problems for their specific qualifications and the strategies they employ. It’s a detailed look at the balancing act involved in creating fair and reliable assessments, and how the CASLO approach, while aiming for specific outcomes, needs careful management to avoid these potential bumps in the road. It’s a reminder that assessment is a complex, human endeavour, and continuous reflection is key to getting it right.
