If there’s one thing that is clear from our EMA landscape review it is that we are facing an awful lot of challenges. It is reassuring to have some clarity about what the issues are and to find that the same issues crop up right across the sector but we need more than that if we are to come up with an effective action plan. Over the last few weeks we have been seeking opportunities to discuss how you view the challenges and what you see as the priorities for action. Key to this process has been engagement with around 80 practitioners via the ALT-C conference and the Jisc Learning and Teaching Practice Experts Group. Naturally, with such a long list and some overlap between the issues, we saw some differences in individual priorities but overall there was quite clear consensus about the top priorities. Below you can see the top 10 challenges mapped against the assessment and feedback life-cycle (click on the image to see it more clearly).
We are trying both to understand your priorities and to establish which of the challenges it is appropriate and feasible for this community to tackle. We had already begun to think that the real solution-finding focus of this EMA programme was likely to fall into the elements of the lifecycle from submitting onwards. This is partly because this is the area where the technical issues are most prominent and also because Jisc has already undertaken activities leading to the production of a wealth of resources to support the earliest stages of the life-cycle. Resources from the curriculum design programme, the managing course information programme and the assessment and feedback programme are already available and will provide valuable input to our toolkit to support the whole of the life-cycle.
Looking at the top 20 challenges overall brings even more clearly into focus the fact that many of the current issues relate to marking and feedback processes (click on the image to see it more clearly):
As we move into the solution finding phase of activity, we will be seeking particularly to engage with those of you who have ideas that can address these issues.
For those who like lists here is the top 20 in the order that you ranked them:
- Ability to handle variety of typical UK marking and moderation workflows
- Reliability of submission systems
- Lack of interoperability between marking systems and student records systems
- Need to develop more effective student assessment literacies
- Student engagement with feedback
- Risk aversion
- Ability to manage marks and feedback separately
- Academic resistance to online marking
- Need for greater creativity
- Ability to gain longitudinal overview of student achievement
- Need to offer greater formative opportunities
- Ability of systems to handle off-line marking
- Systems not geared to peer assessment
- Ability of systems to support variety of moderation process
- Differences of opinion on value of anonymous marking
- Notifying students when feedback is ready
- Ability to handle variety of anonymity requirements
- Ability of systems to support variety of grading schemes
- Systems not geared to group submissions
- Ability of systems to deal with mathematical, scientific, musical etc notation
The first thing that strikes me is that the priorities show us still – even after all these years – bottlenecking at the technical/functional level of workflows, platform reliability and interoperability. Only at do priority 4 do more educationally ambitious needs start to be represented. I can understand this order of priorities – it looks like a Maslow’s hierarchy of needs thing (is there a TEL equivalent? Beetham & Sharpe’s digital literacies framework?)
What this suggests to me is that a majoritarian approach to prioritsation may continue to divert us from assessment pedagogies. If we want to avoid this, one thing we could try is to consider each priority from three perspectives – value to the sector, complexity, and effort. This might yield a different order (though it might not).
But the list does clearly reveal the impediments that we as a sector have to scramble over to be able to start to pursue the pedagogical ideals of e-assessment. It feels like a big step forward to have these priorities – thank you to all involved.
Pingback: Assessment & feedback – links from the Joint Faculty E-Learning Forum | UCL E-Learning Environments team blog