2 Increasing student engagement in their learning through active collaborative learning

Introduction

This chapter provides an overview of scaling up and evaluating Team-Based Learning (TBL) as an active collaborative learning approach at Anglia Ruskin University (ARU) over three academic years (2015 to 2018). After an introduction to the institutional and UK Higher Education context and the rationale for introducing TBL at ARU, the chapter discusses the evaluation as part of an Office for Student (OfS) Catalyst Project, including methodology, results and concluding with recommendations.

What is TBL?

According to Sweet (2010), TBL is a

TBL is an active learning methodology suitable for a range of disciplines, and one, which can readily be understood and adopted by teachers (Morris, 2016). It provides a flexible framework into which other pedagogies can be incorporated, such as problem-based learning, enquiry-based learning, for example. It offers rich formative feedback throughout a course (called programme elsewhere), from peers within each team and across teams, as well as from tutors.

TBL is a structured approach, which combines flipped or inverted learning with team-based in-class activities. Materials are provided to guide independent pre-class learning, covering the core concepts, theories and models for the topic. Pre-class learning is assessed via the Readiness Assurance Process, which involves both individual (iRAT) and team (tRAT) multiple choice tests followed by tutor feedback to address gaps and misconceptions identified by the tests. Teams then carry out activities in which they apply concepts and theories to practice (Application Exercises). These activities use what is called the 4S approach where the activity tackles a Significant problem; teams work on the Same problem; make a Specific choice; and report their results Simultaneously. The tutor strategically assigns students to teams at the beginning of a module or programme which remain constant throughout the semester/year.

TBL was initially developed by Larry Michaelsen in the late 1970s as a response to the need to move from small to large group teaching. TBL became more established in the 1990s and spread worldwide as well as from predominately business-related disciplines and medical and related areas to other subjects (Michaelsen et al., 2002).

TBL caters for ARU’s mission to support a diverse student body and increase attendance, participation and engagement. It does this through the accountability to teams, the use of frequent summative tests of pre-class learning (i.e. iRAT/tRAT), followed by authentic learning activities supporting active and deep learning (Michaelsen and Sweet, 2009).

In the past 20 years research into the effectiveness of TBL has increased, demonstrating improved educational outcomes (Haidet et al., 2014). However most studies focused on pilots or case studies and there is little research evidence about the routine or institutional use of TBL (Sisk, 2011; Burgess et al., 2014; Dearnley et al., 2018).

Institutional and UK Higher Education context

ARU is a modern university having received university status in 1992. It has four main campuses and a number of regional partners located in the East of England as well as international partners serving a diverse population of approximately 39,000 students. ARU has ‘invested heavily in recruiting and supporting a student body that draws students from a wide range of backgrounds. As a result, ARU has one of the most equitable mixes of undergraduate students of all UK universities’ (ARU, 2018: 1).

ARU’s University Strategy, Designing our Future 2017–2026, includes commitments to ‘provide an inclusive, stimulating and innovative curriculum which supports our diverse student population [and to] attract, engage, challenge and empower students from a wide range of academic and societal backgrounds to reach their full potential’ (ARU, 2017: 3). Consequently, the university promotes staff development that is ‘focused directly on our students’ educational experience’ (ARU, 2018: 2). Among other measures, staff development has focused on active collaborative learning practices, and an institution-wide drive to shift the balance of feedback away from summative and towards formative feedback and feed forward at an earlier stage. The learning and teaching provision at ARU aims to ensure success in measures such as student attainment, retention and employability to successfully compete in the UK Higher Education landscape.

Active Collaborative Learning: TBL

As part of improving student performance, attendance, satisfaction, and ultimately retention and employability, ARU has introduced a number of active learning approaches into the curriculum. One of these approaches, TBL, was introduced to ARU staff through staff development led by Professor Larry Michaelsen (University of Central Missouri) and Dr Simon Tweddell (University of Bradford) in the 2014/15 academic year, which was followed by a number of pilots in 2015/16. Staff development continued to be provided internally and more staff were encouraged to adopt TBL across all disciplines in a push to scale-up active collaborative learning. There was strong senior management support, which is reflected in institutional policies and strategies as well as investment.

Based on the success of the TBL implementation, funding was secured in 2017 from the OfS Catalyst Project, Scaling up Active Collaborative Learning for Student Success (NTU, 2019), with two other universities, Nottingham Trent University (NTU), which was the lead institution, and the University of Bradford (UoB), to scale-up active collaborative learning activities. This meant TBL at ARU and UoB, and SCALE-UP (Student-Centred Active Learning Environment with Upside-down Pedagogies) at NTU (NTU, n.d.) and evaluate their impact with a specific focus on students from groups that frequently underperform in higher education.

Methodology

Starting in 2015/16, we evaluated our TBL pilots using a mixed methods approach of student and staff surveys (Semester 1, 2015 and 2016), and semi-structured staff interviews (Semester 2, 2016).

As part of the OfS Project, we evaluated TBL more widely at ARU in conjunction with our two project partners using student and staff surveys and student data on attendance, engagement, satisfaction and performance for each module and course. We compared TBL modules with non-TBL modules in courses including at least one TBL module. Starting from our initial research, we also identified barriers to scaling up TBL at ARU and identified and developed solutions to address these challenges (see Chapter 7). This was an ongoing piece of research whereby the original list of barriers was updated when new barriers were identified and/or solutions found.

Adoption was evaluated in the 2017/18 academic year against a typology of constituent elements of TBL to identify various combinations (Berkson and Richter, 2018). While most adoptions followed the original TBL approach, there were variants in the elements used (e.g. the classroom may not have been flipped or iRATs/tRATs were only used formatively). A further variation was the duration, with a small number of modules only applying TBL in some sessions and not throughout a module.

Results

In the following sections, we summarise the main ARU results from the different student and staff evaluations of the OfS project. There were a number of common results across different methods of evaluation, so we have combined these findings. The majority of questions in the two student questionnaires in 2015 and 2018 were similar, while some questions in the 2018 version were included to accommodate the different active collaborative learning approaches and environments of our project partners. The questionnaire results provided differentiated and deeper insights into the student experience with TBL. The 2018 staff survey focused on defining the variations of TBL based on a typology but also included some qualitative questions relating to the experience of staff with TBL.

The biggest set of data was the comparison of TBL against non-TBL modules within a course looking at TBL modules alongside non-TBL within a year and across three years (2015 to 2018) for student performance (i.e. module marks, course completion rates, and pass marks), attendance and satisfaction (based on Module Evaluation Surveys (MES)) as well as engagement by modules (based on a combination of attendance, library and virtual learning environment use). The numerical data also allowed us to drill down to results by groups of students defined by gender, age (i.e. under and over 21 years), POLAR data (i.e. postcode), minority groups, and disabled students.

Extent of Adoption of TBL across ARU

To identify modules using TBL across the institution, we used a combination of timetabling information (i.e. use of Active Learning rooms), reports from faculty Directors of Learning and Teaching and TBL Leads, and sometimes anecdotal information. Confirming an accurate number of TBL modules proved to be a challenge as teaching methods are not coded on any university systems, which meant reliance on personal contact, and often chasing information to establish the real extent of TBL adoption in each of the faculties. To identify the number of courses including TBL modules, we used a data extract from the student administration system and filtered by undergraduate courses with TBL modules delivered on ARU’s main campuses. Over the past three years (i.e. 2015–18), ARU has seen a steady increase of TBL at module level from an initial 25 modules in 2015/16, mainly in Business and Sciences, to 32 in 2016/17 (+28 per cent), and 38 modules in 2017/18 (+18.8 per cent) across different disciplines.

Adoption by course increased between 2015/16 and 2016/2017 but dropped in the following year. However a number of large modules are shared across several courses. Some of these modules stopped using TBL in 2016/2017, which explains the reduction in the number of courses in that year, while the number of modules increased (see Figure 2.2). Discontinuation of TBL can be attributed to a number of different causes, ranging from discontinuation of the module altogether, to a change of module leader or tutors, and sometimes as a result of changes in the re-approval process.

Quality of Adoption against the TBL Typology

We sent a survey to 28 module leaders, who were identified as running TBL modules in 2016/17 and received 19 responses (68 per cent) regarding 30 modules (some module leaders ran multiple TBL modules with similar approaches). The survey focused on identifying the format of TBL taken by the different modules using a TBL typology based on a maturity model with higher maturity attributed to higher levels of student-centred and independent learning. The spider diagram in Figure 2.3 illustrates the adoption across all 19 responses.

Most tutors use TBL-type application exercises and broadly adopt a 4S approach. There are, however, variations between different modules with many lecturers not truly flipping the lecture, but instead using a lecture or just reading in place of pre-work, or mixing TBL with other active learning approaches. Another frequent variation is a mix of formative and summative TBL assessments.

Another variation concerned how TBL was adopted in modules with the emergence of ‘occasional’ TBL as new phenomenon. In these instances, module leaders adopted TBL only for a few sessions rather than across the whole delivery, which meant that the positive effects of forming permanent teams was limited. The figures in Figure 2.4 below illustrate how three categories of variations we identified performed for attendance and module mark. The term ‘Half-TBL’ described modules with either a mix of TBL with other approaches, or elements of TBL such as application exercises were used, but the lecture was not flipped. Figure 2.4 also shows how a particular TBL variation performs on module marks with a full(er) adoption performing better than the other two against the average module mark of non-TBL modules. Students tend to perform worse in modules where some elements of TBL were not adopted. This suggests that the limited or occasional adoption of TBL during a few weeks in a module is unlikely to have a significant effect on module performance as performance is based on the dominant approach used to teach these modules.

Figure 2.5 illustrates how the adoption of TBL influences attendance. While those modules being mostly or fully TBL have above average attendance against the average attendance of non-TBL modules, the other two categories perform similarly, with the attendance figures for occasional TBL showing significant variation in itself. The higher attendance in mostly or fully TBL modules suggests that cohesion and positive group dynamics develops in teams. The accountability of individual team members for the team performance can result in higher attendance.

Student Satisfaction

For the student satisfaction questionnaire 2018, we approached all students taught using TBL in the 2017/18 and 2018/19 academic years with a request to participate in an online questionnaire to evaluate their experience with TBL. Module leaders were also asked to encourage students to respond to the questionnaire, for example via announcements on the Virtual Learning Environment and allowing class time for completion.

A total of 327 responses were received from 231 students on 34 modules (some students were taught TBL on more than one module) across four of the five faculties, which is the equivalent of a 6 per cent response rate. The number of responses for each TBL module varied from low (n=1) to high on two modules (n=111 and n=118). The majority of responses (n=229) came from two large nursing modules.

The survey included 24 blocks of questions, most of which were closed 5-point Likert-style questions, asking students to state the extent to which they agreed with various statements (i.e. strongly agree, agree, undecided, disagree, strongly disagree), one question asked for satisfaction (i.e. very satisfied, satisfied, neutral, dissatisfied, very dissatisfied), and a few for applicability (i.e. yes, yes/no). The latter often linked to an open question to elicit more details. The questionnaire also included two general open questions at the end.

A two-thirds majority of respondents was satisfied with their TBL experience and agreed that the different aspects of TBL have a number of benefits and advantages over other learning approaches. For instance, students agreed that TBL promotes employability and is a more inclusive way of learning. However, between one-fifth and one-quarter (depending on the question) were either neutral or critical about TBL. While there were always critical comments, the number has increased from the 2015/16 evaluation reflecting the change from pioneers to early adopters in scaling up TBL. Recurring themes were students’ preference to work independently rather than engage in teamwork, the need for a clear induction into TBL, clear instructions and communication, as well as scaffolding, which also relates to constructive alignment (Biggs, 2014) between the different elements of TBL (i.e. pre-learning, iRAT/tRAT, and application exercises) and assessment. Aspects of delivery such as consistency between tutor teams in co-taught modules, and time requirements for pre-learning being considered across a course rather than at module level also have scope for improvement.

TBL versus non-TBL modules: Attendance, Engagement and Satisfaction

As part of the project, we analysed data extracts from the student administration and other systems to compare TBL and non-TBL modules and courses including TBL modules over three academic years. This allowed us to investigate the performance of students on TBL modules as opposed to non-TBL modules within an academic year, and modules and courses across three years.

We found that attendance on TBL modules was higher over the three-year period than on related non-TBL modules, where ‘related modules’ refers to non-TBL modules in a course with at least one TBL module. The engagement score improved when students took a single TBL module, and TBL did result in measurable increases in satisfaction scores. However, the latter is not unexpected as the questions in the MES are not specific for TBL, and we did not have access to the free text comments. From the previous evaluation, we found that these comments were supportive of TBL.

Figure 2.6 illustrates the improvement of module attendance we found over the three years. Engagement, which is predominantly based on attendance (60 per cent), also improved over the three years (see Figure 2.7).

Both the attendance and engagement figures show less of a difference in 2015/16 which was most likely due to a mix of manual capture, such as registers, and a newly installed automated ‘tap-in’ system, neither of which produce reliable results.

TBL versus non-TBL modules: Marks

Over all three years of the study, students achieved slightly higher (2–5 per cent) marks in TBL modules than non-TBL modules (see Figure 2.8). The improvement in average marks for the TBL cohorts was matched by an improvement in the pass rate for the modules, with 1–5 per cent more students passing their TBL modules than non-TBL modules (see Figure 2.9). This fits with evidence in the literature (Koles et al., 2010) showing that lower performing students benefit from implementing TBL.

Conclusion

Overall, the results of this research clearly demonstrate that the introduction of TBL at ARU had a positive effect on students’ learning behaviour and outcomes. As with any adoption and scaling-up of a new learning and teaching approach there are barriers to overcome to progress from the enthusiastic pioneers to early adopters and late adopters. The barriers and the solutions are discussed in Chapter 7.

A number of recommendations can be drawn specifically from the student satisfaction questionnaire:

• We need to acknowledge that variations in adoption of TBL influence outcomes. TBL is often not used out of context but tends to be used in combination with other active learning approaches. To scale up active collaborative learning and TBL it is recommended to take a course level approach.
• Consistency of delivery across modules, which are co-taught by different tutors, is important not just for TBL to ensure students get an equivalent learning experience.
• When teams are formed ensure that they are balanced as the strength of a team is in the diversity of its members.
• It is important that different team members contribute equally to team activities. It is worth considering getting team members to agree expectations for their team at the beginning (Eberly Center, 2016; Riordan and O’Brien, 2012).
• Often a ‘managed’ change of learning culture is required to motivate students to become team players rather than competitive individuals.
• Pre-learning, application exercises, and assessment need to be well constructed and scaffolded to gain the full attention of learners and avoid distraction.
• Pre-learning activities and application exercises need to be varied to cater for a wide range of learning preferences and patterns.
• Pre-learning, iRATs/tRATs, application exercises, and peer assessment need to be constructively aligned to each other and to the assessment.
• The time allocated for pre-learning tasks needs to be realistic and related to the workload of students across their courses as well as taking into account that many students have competing demands on their time (such as working, child care, caring for relatives).

Active learning is here to stay, and as an institution ARU has made significant inroads into changing the culture of learning and teaching. We have identified a number of challenges and are working to overcome these to continue scaling up TBL and other active learning approaches.