When we talk about assessment, especially in educational contexts, it's easy to get bogged down in the technicalities. But at its heart, it's about fairness and accuracy, isn't it? The CASLO approach, as explored in a recent report, isn't immune to scrutiny. In fact, the report delves into potential criticisms, and understanding which ones hold the most weight is key to improving how we evaluate learning.
Looking at the findings, the most frequently acknowledged potential problem by those involved – the Assessment Organisers (AOs) – was 'inaccurate judgements and standardisation.' This isn't a minor quibble; it speaks to the very core of assessment. The literature suggests that the criteria used in CASLO qualifications can be tricky to write and interpret precisely. This ambiguity, the report highlights, can make it difficult for assessors to consistently determine if a student has met the required standards, or even to differentiate between grades. Imagine the frustration for both students and educators if the goalposts feel unclear or if the grading can feel a bit like a lottery.
This particular criticism resonates because it directly impacts the validity of the assessment. If judgements are inaccurate, then the whole process of measuring competence or achievement is undermined. The report notes that for 12 out of the 14 qualifications sampled, this was seen as a potentially relevant issue. Even when AOs didn't see it as a current problem, they often discussed mitigations they put in place, suggesting an awareness of the inherent challenges.
While other criticisms were raised – like 'atomistic assessor judgements' (which was the least recognised), 'poorly conceived assessment tasks,' 'lenience and malpractice,' and 'inappropriate support' – the issue of inaccurate judgements seems to be the most pervasive and fundamental. It's the bedrock upon which all other assessment activities are built. If that foundation is shaky, the entire structure is at risk.
Interestingly, the report points out that the responses from AOs regarding mitigations didn't drastically differ whether they recognised a problem or not. This suggests a proactive approach to managing potential risks, even if the perceived severity of those risks varied. However, the sheer prevalence of 'inaccurate judgements' being recognised as a potential issue, coupled with its direct impact on the fairness and reliability of assessment outcomes, makes it arguably the most valid and significant criticism of the CASLO approach as presented in this research.
