We as a team – Martin Wright, Tim Fewster and Ksenia Shalonova – did live marking for the Foundations of Statistics data analysis reports (1,000-1,500 words). We present two blog articles: the first gives a brief description of the background information and summarises both the advantages and disadvantages of using live marking, whereas in the second, Tim Fewster shares his personal experience about using live marking for the first time. We hope that Tim’s blog article will inspire you to use live marking … in particular, will inspire those of you who are reluctant to use it in your sessions and for your subjects – as we as a team (the three of us) enjoyed the whole process.
Live marking in the AI era – a new strategy for engaging students?
Why live marking?
A traditional way of marking technical reports involves a tutor providing both grades and written feedback to students by communicating them using online platforms. A recent trend is to explore AI-powered tools to facilitate marking and feedback, but currently such an approach can potentially do more harm than good (Taylor, 2024).
Live marking is a method of providing both marking and feedback in real time, talking face-to-face with students, and can be done both for formative and summative assessments. Live marking has some similarities to oral exams, as it involves a verbal dialogue in which students have input in their assessment and can respond to feedback immediately, asking questions where they feel appropriate.
Moreover, direct and explicit instruction that a tutor provides in the live marking sessions can be considered more effective in learning than minimal guidance. Kirschner & Neelen (2016) found that the explicit instruction and feedback drives students’ motivation and engagement. This is particularly true for teaching mathematics for novices and intermediate learners, where achievement in mathematics has a positive effect on motivation and engagement.
What was done?
The team (Tim Fewster, Martin Wright and Ksenia Shalonova) were delivering the IFP Foundations of Statistics unit (35 students), and live marking sessions were run for the formative drafts of their data analysis reports (the summative = 30% of the unit assessment). The three tutors organised their sessions in a slightly different way, but the main difference was that two tutors had pre-read the reports in advance, whereas one tutor was doing live marking in real time. Hence, one tutor suggested that the whole process should be called live feedback, rather than live marking.
Tutor experience
From the tutors’ perspective, the following advantages of live marking were identified:
- It helped tutors to better understand students’ (mis)conceptions in statistics
- It allowed conversations about students’ understandings of statistical concepts
- It was an opportunity for students to learn further during the sessions
- It helped develop students’ communication skills
- Students were able to act on the feedback immediately
The following disadvantages of live marking were identified as well:
- Live marking is not a time saver and can be even more time consuming if tutors prepare for the conversations
- It can increase mental strain on tutors when reports are not read in advance
Student experience
Survey data from the students (50% response rate) shows that they all strongly agreed with the statement that live marking was very helpful for learning and engagement compared to traditional methods.
‘we can immediately give questions about improving our work’
‘she evaluated my report using the marking criteria, allowing me to understand which points I need to improve on’
‘can ask questions promptly’
‘helped to actually understand the feedback and where the improvements in my work’
‘It’s much clearer to see where to correct and improve; we can ask questions about further advice’
‘Help understand and review my coursework more in depth and able to correctly identify my mistakes. To talk face to face with the tutor’
‘The feedback is really detailed and it is really convenient that I can ask my tutor directly about aspects I’m not clear.’
‘Very specific feedback, i was told exactly what to do and not do’
The way forward
In the era of generative artificial intelligence (AI), educators face challenges in designing assessments that both evaluate authentic student learning and engage students in the learning process. It is quite clear that rethinking assessment strategies will be inevitable. One of the solutions could be introducing live marking for the summative assessments as well, for example, for the IFP mid-unit maths exams, as well as for the summative reports. The assumption is that the learning achievements in mathematics, supported by live marking, can impact students’ motivation and engagement.
References
Kirschner, P. A., & Neelen M. (2016). “Close the Stable Doors: Effects of Motivation and Engagement on Learner Achievement?” Available at: https://3starlearningexperiences.wordpress.com/2016/05/17/close-the-stable-doors-effects-of-motivation-anengagement-on-learner-achievement/
Taylor P. (2024). “The Imperfect Tutor: Grading, Feedback and AI.” Available at: https://www.insidehighered.com/opinion/career-advice/teaching/2024/09/06/challenges-using-ai-give-feedback-and-grade-students
Some thoughts on Live Marking in Statistics
by Tim Fewster
As a new member of the team at CALD, and someone whose background is teaching Maths in secondary schools, I wasn’t thrilled to discover I would be marking data analysis reports for my Foundation of Statistics group. I was even less excited to hear that this year we would be trialling live marking. I knew that live marking had become something of a trendy pedagogical approach in schools in recent years, but as a Maths faculty [in a secondary school], we had largely managed to avoid it. We had argued quite successfully that we already do it in Maths. The nature of our subject means there are lots of quick, easy to mark answers and we get the students to do it themselves whilst we circulate and help those who make errors. I felt a certain satisfaction as I heard about what teachers in the more text-based subjects [at our secondary school] were being asked to do – namely, find something to keep 29 often uninterested and noisy teenagers quietly engaged whilst they sit down with one pupil and spend five minutes marking a piece of text that they have just written and give some detailed feedback.
So it was with some trepidation that I said I would happily do some live marking at CALD and then write about my experience for the blog. First a bit of context; Foundations of Statistics students are assessed by an end of unit exam (70%) and a data analysis report (30%). The report is 1000-1500 words long and is a statistical analysis of one of 3 datasets that we provide, with students coming up with their own research question and then investigating it. They submit a draft report in week 14, we give them some feedback and then the final submission is due in week 20.
As the deadline for the draft report edged closer and I became familiar with exemplar reports from previous years, I started to realise that marking completely live without pre-reading the reports would not be an option. I was too unfamiliar with the marking framework to just do it off the cuff and I didn’t feel remotely confident that I’d be able to give students the feedback they needed. So, the plan changed slightly from live marking to live feedback. Once the reports came in, I read and annotated them with my feedback and at the same time put a summary of this into a word document that also contained the RAG feedback table containing the assessment criteria. Then in the week following the submission, I took IFP students out of the class one at a time and sat with them to discuss their draft report. There was – unsurprisingly – a huge range in how complete their reports were at this stage. This meant that some students were getting feedback based around a few minor improvements, whilst some were being reminded of the basics of what they needed to do all over again. I had planned to spend about 10 minutes with each student but – as is often the way with these things – the sessions tended to overrun. I talked the Foundation of Statistics students through their report and what improvements could be made, then I filled out the RAG feedback table for them. At the end of the session I emailed them this document. The feeling by the end of the 2-hour live feedback session was one very similar to how I used to feel at the end of a parents evening in school: in need of a lie down in a dark room with a cold drink.
Despite feeling a little drained by the process, from my perspective as the teacher, the conversations felt very productive for the students. The fact that they can ask for clarification on my feedback has obvious benefits over giving them some written feedback to take away and ponder in their own time. The conversation often went off in different directions as it became apparent the student didn’t fully understand a certain statistical method … so then we would open up Excel and I could demonstrate what to do. I would say they all left the session with a clear idea of at least one thing they needed to do to improve their report.
So that all sounds great – but what were the issues? Logistically it was tricky, and timetabling meant that some students got their feedback on a Monday whilst others had to wait until Thursday. I foolishly didn’t think too much about what the other students were doing whilst feedback was being delivered. Those who had already had their session could obviously crack on with their report – but if they were waiting to sit with me, they had kind of reached a natural point where they didn’t have much more they could do. This was flagged by students when we got feedback from them, and someone suggested giving them some mixed exam questions to practice, which is a nice easy fix for next time. It was also tricky to fit it all into the timetabled classes for the week, so I wonder if it could be done outside of lessons. Perhaps it could be carried out in a similar way to exam paper viewings with students booking in slots on a Wednesday afternoon.
Overall, I found it to be beneficial for the students and for me as a teacher. Following a survey, the students were also very positive about the experience, with 100% of the respondents saying they found it very helpful. The only suggestions for improvements were to have more time with the teacher and/or to have two separate live marking sessions for each student at different points in the report writing process. If I’m teaching this again next year, I would make a few changes, but I think I would still be giving live feedback as opposed to actual live marking. Ksenia was far braver than me: she went into her sessions without pre-reading the reports at all. Having spoken to her shortly afterwards, I think I am glad I did it the way I did.