The right tools for the job

Approaches to marking and feedback are often highly personal and conditioned by preferences for online or off-line marking. Some staff prefer familiar tools such as Microsoft Word but find the lack of integration with other EMA systems a drawback (e.g. the need to return work to each student separately – involving an email per student). There are many efficiencies in online marking through integrated tools such as Grademark and tools available in the commonly used VLE systems with clear time savings on the overall process (regardless of how long it may take individuals to mark by these means as this is largely dependent on their familiarity with the system). Deciding which tools to use it is not necessarily a straightforward matter if the institution supports multiple tools as the attractiveness of each may vary according to the type of assignment and marking process.

Below is a very useful comparison of two commonly used tools (Turnitin and Moodle). We are grateful to Dr Mira Vogel and colleagues at UCL for providing this analysis and we would be interested to hear from others who have undertaken similar comparisons.

In her context Mira concludes:

The major advantages mentioned by staff who use Turnitin are the ability to create one’s own bank of frequently-made comments (QuickMarks), the ease of clicking these into place, and the ability to position these so as to closely relate them to the relevant part of the essay. The seamless progress through the different submissions without having to save, close, and open the next is also valued. There is also the Similarity Report which shows how much of the work is original and how much is from other sources.

The major advantages mentioned by staff who use Moodle Assignment are being able to download, mark offline and sync the markings back to Moodle, the support for multimedia and larger file sizes, and the ability to upload multiple feedback files. The ability to submit as a group and receive a mark in common is relatively new, but promising.

In both cases it is possible to incorporate existing standardised mark sheets or rubrics to expedite the feedback process.

Comparison between Turnitin and Moodle assignment, organised by process

Capability Turnitin Moodle Assignment
Overview Third party, commercial software integrated with Moodle. Core to Moodle (the open source Virtual Learning Environment we use at UCL).
Supported formats

Essays

Yes. Yes.

Multimedia

No.But coming soon. Yes.

Maximum file size per submission

20Mb. 160Mb.

Number of separate files

Up to 5. Up to 20.Or limitless if files are ‘zipped’.
Operational processes

Impose deadlines / prevent editing

Yes. Yes.

Feedback on drafts

Yes.Caveat – ask ELE to assist with settings to avoid resubmissions overwriting earlier feedback. Yes.Caveat – ask ELE to assist with settings to avoid resubmissions overwriting earlier feedback.

Pastoral activities (eg follow up on late work)

Yes. Yes.

Release or withhold marks and feedback.

Yes.Caveat – not on an individual basis. Yes.Caveat – not on an individual basis.

Word count

Yes.Caveat – inclusive of coversheet, references, etc. Yes.Caveat – yes if word processed, no if PDF.

Download summary report on spreadsheet

Yes. Yes.
Governance

Blind marking (student anonymity)

Yes.Caveat – if marking is shared between more than one marker, arrange who marks which submissions, and how to identify them (e.g. apply Moodle Groups or sort by paper ID). Yes.Caveat – if marking is shared between more than one marker, arrange who marks which submissions, and how to identify them (e.g. apply Moodle Groups or sort by paper ID).

Double blind marking (multiple markers working in isolation)

Working solution.Caveat – workaround necessary to present each marker with clean submission. Working solution.Each marker downloads the submissions.

Moderating

Yes. Yes.

External examiners

Yes.Caveat – see Double Blind, above. Yes.Caveat – see Double Blind, above.
Marking and feedback

Similarity Report

Yes.Caveat – very helpful for students to have the chance to respond to this before the deadline (easy to set up). It is also important to negotiate equitable marking processes around the reports. ELE can make suggestions. In short, it’s simple matching that Turnitin does, so any coversheet will register as a match, for example. No.But possible to support students to use Turnitin independently to check their referencing, paraphrasing and quoting prior to making a final submission.

In-context (bubble) comments

Yes.Type your comments directly into Turnitin, drag them to where you need them, edit as needed, and save to use again, if helpful. Yes.If open and mark in Word Processor or PDF reader software. Unlike Turnitin, no easy way to save for future use.

Structured mark sheet and/or rubric

Yes. Yes.

Freehand drawings

No Yes.If open and mark in Word Processor or PDF reader software.

Spoken word feedback

Yes.Record directly into Turnitin. Potentially. No voice comments in MS Word, but possible to upload any kind of feedback file to Moodle.

Track changes

No. Yes.If open and mark in Word Processor or PDF reader software.

Feedback files accepted (i.e. different media)

Yes. Yes.

Rubric or structured mark sheets

Yes. Yes.

Check if students have accessed feedback

Yes. Yes.In a more roundabout way than Turnitin.

Offline marking

Yes, only using iPad app (with caveats). Yes, bulk download and bulk upload of response files.

Bulk download

Yes.Caveat – if anonymous marking is enabled then this is only possible only after the Post Date. Yes.
Group work

Apply Moodle groups.

Yes.I.e. to connect markers to pre-existing groups of students within a cohort. Yes.I.e. to connect markers to pre-existing groups of students within a cohort.

Collaboration

No.No shared editing environment. No.No shared editing environment.

Submit as group

No. Yes.

Differentiate between individual contributions.

No. No.

 

6 thoughts on “The right tools for the job

  1. Roy Currie

    The Moodle GradeTracker plug-in (developed by Bedford College and freely available from http://moodlegrades.bedford.ac.uk) provides a tool for tracking progress against BTEC, C&G and A/AS Level qualifications that fully integrates with Moodle assessments and activities. When setting on-line assessments, teachers can specify the units and criteria evidenced in the assessment and learner tracking grids are automatically updated on submission (or passing of deadline). The system also displays predicted grades based un performance within units against targets calculated from prior learning and ALPs weightings. Teachers can also export/import offline marking spreadsheets that support easy provision of feedback at criterion level.

    Reply
  2. Roger Gardner

    Thanks, Gill, for this and the other postings which I have found extremely useful and interesting reading.

    One comment I often get from academic colleagues exploring online marking approaches is that not only is the length of assignment an important factor but also the size of the cohort. They may be happy marking 20 x 3000 word essays online, but not 200.

    Secondly, for some, their preferred tool/approach for reading the work may be different to the tool for providing feedback e.g. reading a paper copy then annotating or providing feedback electronically.

    Reply
  3. Pingback: 5. Marking and production of feedback | Electronic Management of Assessment

  4. Pingback: The right tools for the job | Electronic Manage...

  5. Marius

    Couple of things that may need rectifying:

    For Moodle:
    Statement: Release or withhold marks and feedback. Yes.Caveat – not on an individual basis.
    Fact: it does work on an individual basis when using the marking workflow in Moodle.

    For Moodle:
    Blind marking (student anonymity)
    Statement: Yes.Caveat – if marking is shared between more than one marker, arrange who marks which submissions, and how to identify them (e.g. apply Moodle Groups or sort by paper ID).
    Fact: Using the marker allocation in Moodle you can asign markers to papers. If blind marking is enabled, one could still identify from the db who are they and allocate accordingly (if applicable or necessary)

    For Moodle
    In-context (bubble) comments
    Statement: Yes.Type your comments directly into Turnitin, drag them to where you need them, edit as needed, and save to use again, if helpful. Yes.If open and mark in Word Processor or PDF reader software. Unlike Turnitin, no easy way to save for future use.
    Fact: Correct but not complete. Moodle comes up with a PDF annotator which allows bubble comments without the need to download – upload. It does not allow saving the comments but it saves time by avoiding the download-upload process.

    Reply

Leave a Reply

The following information is needed for us to identify you and display your comment. We’ll use it, as described in our standard privacy notice, to provide the service you’ve requested, as well as to identify problems or ways to make the service better. We’ll keep the information until we are told that you no longer want us to hold it.
Your email address will not be published. Required fields are marked *