he EMA Requirements Map working group is looking at common workflows. Discussions about how many models of marking exist have been going on for a while (see earlier post How many models of marking are there?).
The key remaining question seems to be whether moderation of multiple markers’ work as opposed to moderation of a single marker actually constitutes a different model.
Point of View 1 – these are not fundamentally different models i.e. they serve exactly the same purpose and have the same outputs. Cohort size is the key parameter here that determines how the model is carried out i.e. whether by one person in each role or multiple people in each role. (indeed for each of the models the same role could have 1 or many actors).
Point of View 2 – these are different models because with multiple markers the emphasis on consistency is much greater and hence the technical requirements are different. The moderator needs to (a) view the range of marks and watch that say all fail grades or all ‘50s’ across various markers are consistent (b) the moderator needs to see a display of grades by marker and the distribution of marks by marker. Technically this is more complex to achieve than a simple case of a moderator moderating the marking of a single marker.
This is an interesting point of debate with implications for how we define an academic model. Are they variants of the same thing with different parameters or are they different in ways that are academically significant? We would like your views ASAP.
Below is a rough representation of this workflow with some of our comments (click on the image to expand it or download the pdf). Let us know what you think.