Thinking around task design on the pre-sessional (summer of 2023)

by Grant Hartley and Tony Prince

Addressing a need

CALD’s pre-sessional courses attracted around 580 students this past year, with roughly 80% studying online, all with the hope of being better prepared for their PG or UG courses at the University of Bristol. As such, the focus of the course is on providing students with an experience where they can become more used to the demands and expectations they will meet in their disciplinary studies. With this in mind, students are taken through a weekly cycle of learning activities, starting with accessing academic texts receptively, before being asked to generate a response to that content. Concepts, tools and strategies which are focused on during this process are then applied to a group project aimed at addressing a challenge within the context of one Sustainability Development Goal.

Within the changing landscape of education, it is vital that CALD’s pre-sessional courses continue to evolve and develop. With this in mind, one of the areas of focus this year was the Academic Reading and Listening Circles (ARC) – part of the work students engage with on Mondays, when accessing academic texts receptively. Feedback from the previous year indicated that these circles were perceived by students as one of the least valuable asynchronous sessions. Part of our attempts at addressing concerns over this aspect of the course was setting up classroom experience data collection of student behaviours during the ARC discussions.

For those unfamiliar with Academic Reading or Listening Circle discussions, these involve students collaboratively working together after accessing a reading or a listening text to share their understandings of a text’s key aspects. This year, students were working with new ARC roles focused more on critically engaging with the texts. These roles were an adaptation of Paul and Elder’s (2022) framework for critical thinking. A further development on this year’s pre-sessional was that the ARC discussions were brought into the synchronous sessions, where previously they had been completed asynchronously. This made them visible and allowed us to look at how exactly these were being done.

This post will outline the process undertaken in the classroom experience data collection with online students and present some reflections on the data collected with reference to possible future developments. It will end by inviting you to respond to this post in the interests of generating a discussion.

Capturing the online student experience

The starting point of capturing the relevant data was to adapt CALD’s teaching observation criteria sheets (see Figure 1 & 2 below) so as to focus solely on student behaviour. Keeping only the ‘Focus on learners’ and ‘Focus on learning’ areas, we shifted the focus of this data collection exclusively to what students were doing in these discussions: their behaviours. It was important to ensure we maintained our focus on student behaviour, as the ultimate aim of this exercise was one of programme development, not teacher development. Communicating this clearly to tutors was essential, to ensure that they did not feel their teaching was being / would be evaluated in this process.

 

Our first task was to focus on what the sections under the ‘Focus on learners’ and ‘Focus on learning’ suggested in terms of student behaviours. This was in part informed by Bavelas et al. (2014) and Keys & Bryan’s (2001) discussions around co-construction of meaning, along with Foster’s (1998) rather dated, but still relevant, discussion on the ‘negotiation of meaning’ (the complete document can be viewed here).

One key aspect of the way in which the data collection was organised was to make use of the cover rota. A data collection schedule was created to assign teachers to specific classes. The priorities in designing this schedule were to ensure coverage of as many classes as possible, while making sure that no class would be visited too often. It was also important to give teachers and students time to ‘settle in’ before their first visit from a member of the data collection team. Those gathering data were asked to keep a running commentary of student behaviour. This method emphasises an objective systematic recording of events, which in turn makes it easier to compare and contrast multiple documents. It also makes it easier to produce a synopsis of the data. Fortuitously, one tutor (thank you Tania Kaya) wrote a synopsis of the running commentary using the boxes provided in the notes section of the ‘Focus on learners’ and ‘Focus on learning’ sections. This proved a useful technique for providing an interpretation of how the data collectors viewed the raw data. While we did subsequently suggest this as a technique, we didn’t want to be too prescriptive as it was initially hoped we could learn from observing different tutors applying different strategies to the data collection.

Until now, feedback on the experience by the data collectors has been limited; however, initial indications suggest the process was generally experienced in a positive light (please add your thoughts in the comment box below this post if you were involved in the classroom experience over the summer). One aspect, which we may need to think more about, is how the form – and specifically the guiding questions – could be informed more by literature, in particular supplementing what we have with insights from the CEFR framework. Engaging with this process collaboratively (perhaps in a Community of Practice) would not only provide more rigour to this process, but may also help develop a shared understanding of what exactly co-construction could involve.

Reflections on what we have noticed, and ways forward

This reflection will focus on one aspect of the ARC activities: online task design. The initial impetus for this focus was on programme development, and it is felt that one key aspect of this is at the level of materials development. One of the key aims of the ARC work is co-construction of meaning. But as the discussion below shows, analysis of the data collected indicates that the current design of the materials for ARC does not always fulfil this aim on the pre-sessional. Revisiting these tasks and thinking around task design variables would seem a positive way forward.

The task design variables could be reviewed along two basic lines of enquiry: input variables and task conditions (Ellis, 2003). Firstly, in terms of input variables, data from running commentaries suggested that the cognitive demands are too high. The cognitive load is perhaps exacerbated for the online cohort by the social distance of the learning environment. In concrete terms, this very often involved students presenting the data related to their role, without a shared screen, significantly increasing the cognitive demand on their group members, who had to remember what was being said. This difficulty was not helped by the focus of the students, which was on ‘narrating’ the notes they had made, without any pause to allow for or encourage questions and comments from others. The question as to what the listening students were doing, and whether they perhaps ‘gave up’, could support Brown et al.’s (cited in Ellis, 2003) observation as to the challenging nature of aural tasks.

The number of elements that students need to manipulate in the current tasks may also be a contributing factor to the seemingly high cognitive load (Robinson, cited in Ellis, 2003). In order for students to successfully address the task, they are currently required to:

• manage the roles
• present their information to other students
• listen to others and make connections to their role
• manage tools used in the online environment

A final input variable contributing to the challenge of these tasks could also be the students’ familiarity with the topic of global citizenship, exacerbated by the level of text difficulty in some cases, as well as length.

In order to address the difficulties outlined above, one of the major design changes to the ARC that is being explored is moving away from a split information task towards a shared information one. This might involve students working on a shared document prior to the synchronous session addressing their role specifically, but which other students can also view. While this may have the added advantage of leading to more asynchronous collaboration at this stage, more importantly, it would ensure students were familiar with the ideas from the other roles before doing the synchronous task. This might lead more naturally into a discussion type task (Willis & Willis, 2007) involving students discussing what the most significant ideas are for the artifact they are required to produce in these sessions. By doing this, the task demands might be reduced allowing for a more interactive experience. Ellis (2003) suggests that shared information tasks can have positive effects on fluency. However, the fact that these tasks will be ‘open’, in the sense that there is no ‘pre-determined solution’, would create a balance in that Ellis does also suggest that ‘open’ tasks appear to go some way in promoting accuracy as well as complexity.

Next steps

We feel the classroom experience data collection has provided us with invaluable insights into student behaviours during the ARC discussions. This has led us to question the task design, and may help us think around ways of moving forward on this aspect of the pre-sessional.

The above ideas are not presented as a ‘fix’ for what we view as areas of concern on the pre-sessional. However, in line with the Think again: Common Read Initiative, we view these as areas to explore through the collection of data, avoiding the pitfalls of the three more limiting mindsets outlined by Adam Grant’s book, ‘Think Again’ (Grant, 2021).

We would like to thank all those who actively participated in this programme development initiative, and in the interests of continuing this discussion, we invite you to comment on, or respond to this post with your thoughts. We would welcome comments from data collectors as well as tutors from whose groups data was collected, or anyone else for that matter. Our hope is that we may develop a discussion around what future iterations of this programme development initiative may look like.

References

Bavelas, J.B., De Jong, P., Smock Jordan, S. and Korman, H., 2014. The theoretical and research basis of co-constructing meaning in dialogue. Journal of Solution Focused Practices, 1(2), p.3.

Ellis, R., 2003. Task-based language learning and teaching. Oxford University Press.

Foster, P., 1998. A classroom perspective on the negotiation of meaning. Applied linguistics, 19(1), pp.1-23.

Grant, A., 2021. Think again: The power of knowing what you don’t know. Penguin.

Keys, C.W. and Bryan, L.A., 2001. Co‐constructing inquiry‐based science with teachers: Essential research for lasting reform. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 38(6), pp.631-645.

Paul, R. & Elder, L., 2022. Wheel of Reason. [Online] Available at: https://community.criticalthinking.org/wheelOfReason.php [Accessed 22 March 2022].

United Nations, n.d. The 17 Goals. [Online] Available at: https://sdgs.un.org/goals [Accessed 4 December 2023].

Willis, D. and Willis, J., 2007. Doing task-Based teaching. Oxford University Press.