7 Barriers to scaling up Active Collaborative Learning

Introduction

After a successful pilot of Active Collaborative Learning (ACL) in 2015/16, Anglia Ruskin University (ARU) joined with two other Higher Education Institutions (HEI), the University of Bradford (UoB) and the lead institution, Nottingham Trent University (NTU), in a national project aimed at scaling up our ACL offerings to institutional level. Like all UK HEIs, ARU needs to ensure that all students fully benefit from their education, which necessitated making the transition from best practice to common practice; from pilots to an institution-wide culture of learning and teaching using proven approaches.

In this chapter we discuss some of the barriers we identified which hinder widespread adoption of ACL. We discuss the solutions we have started to implement at ARU, and compare some of the approaches taken at UoB and NTU.

Scaling up to institutional adoption

There is a strong evidence base for the effectiveness of ACL, including improving student engagement and outcomes in a wide variety of contexts (Freeman et al., 2014). Most literature refers to educational innovations where ACL is introduced in a context where it was not previously being used (Haidet et al., 2014). There is usually a strong motivation to change the learning and teaching approach, championed by enthusiasts. Initial proof of concept studies may apply a quasi-experimental design where cohorts are split, with one group taught using traditional methods, and another group taught using the new method (Deslauriers et al., 2011). In these contexts, the change to ACL produces a clear improvement compared to the previous approach.

However, there are inevitable biases in this kind of educational research. Novelty in its own right can contribute to initially measurable improvements. There is also an element of publication bias where studies that show a marked improvement are more likely to be submitted to and accepted by journals than interventions with limited effects (Franco et al., 2014). We must therefore question whether studies of innovations are representative of the efficacy of a learning and teaching approach in more general use, particularly if the original data comes from an intentionally experimental setting.

Institutions may encounter significant barriers in making the transition from innovations by pioneers to adoption at scale, and must commit to solving these barriers in order to scale up. Following a phase of institutional investment in an innovation, once the new method has been in place for some years, there are likely to be fewer targeted resources available. To be sustainable, the approach needs to be adapted to more constrained circumstances. In addition to overcoming practical barriers, scaling up a method requires a widespread change of organizational learning culture. This change needs to permeate to all levels of an institution, including the students who need to embrace being taught differently from how they may expect. Successful scaling-up is more effective when support is available from the senior managers at the institution.

A particularly interesting question is how well an ACL method performs as it transitions to ‘business as usual’ across the institution. Scaling up the approach presented us with a number of challenges which were not apparent during the initial stage. By analysing some of these challenges and how they can be resolved, we aim to develop guidance for institutions that want to extend an innovation that has demonstrated promising results.

Context of project

In 2017, three universities working in partnership, ARU, UoB and NTU, received Catalyst funding from the former Higher Education Funding Council for England (HEFCE), now the Office for Students (OfS), for a project called, Scaling up Active Collaborative Learning for student success (NTU, 2019). The Catalyst fund was designed to ‘address barriers to success’ for groups of students who ‘experience significant differences in levels of retention, attainment and progression’ (NTU, n.d.). The three institutions had demonstrated success in introducing two ACL methods: Team-Based Learning (TBL) at both ARU (see Chapter 2) and UoB (see Chapter 3), and SCALE-UP at the lead institution, NTU (see Chapter 1). Evidence from the literature shows that these methods benefit all students, but with additional benefit to students who otherwise perform poorly (Beichner et al., 2007; Koles et al., 2010).

The OfS project started in February 2017, at which time UoB had used TBL for over four years, primarily in their Pharmacy course; ARU had used TBL for two years in a number of different courses (known as programmes at some institutions) in the Business School and the Science faculty; and NTU had used SCALE-UP for four years in several departments. All three universities were committed to expanding these ACL approaches across their institutions, in line with strategic plans to increase active learning and effective, innovative pedagogies promoting success for all students. From September 2017, the three institutions embarked on ambitious plans to encourage more modules and courses to adopt ACL, and extend ACL to all faculties and schools/departments.

Methodology

Our approach to identifying and addressing barriers to scaling-up was based on an Action Research cycle (Carr and Kemmis, 2003). Barriers were identified in the original funding proposal, based on data from pilot studies and general experience of educational change. We designed research instruments, including questionnaires and semi-structured interviews, with these barriers in mind, asking staff about whether they had experienced predicted barriers. Colleagues provided feedback on what resources they required to overcome these barriers to successfully scale up ACL. In parallel, we implemented solutions to any barriers we were able to address from the start of the project. We discussed our solutions with stakeholders across the institution and refined them based on the feedback. Ongoing research into barriers, with a mixed methods evaluation of scaling up, allowed us to identify emerging themes and implement or propose further solutions.

Design of research instruments

All three partner universities had successfully introduced ACL in some courses prior to the start of the project, with support from experts in TBL and SCALE-UP. These pilots were evaluated through surveys with staff and students, along with preliminary outcome data. While overall findings were very positive, some challenges were reported by the original pilots, and these challenges were incorporated into the business case of the funding proposal. Description of the expected barriers was refined through predictions of what issues might become more prominent as ACL was scaled up. The project teams undertook detailed discussions with colleagues from all parts of our institutions, as well as applying their knowledge of the institutional context.

As part of the effort to scale up ACL, we started to identify and introduce solutions to the barriers. We included questions in staff surveys and interviews about whether academics had encountered similar problems to those seen by the early adopters, and about what support had been helpful in overcoming them. Results from these evaluations were compared across institutions, identifying patterns that held true in a range of contexts, plus a small number of institution-specific challenges.

The scale of the project provided the opportunity to analyse the experiences of more staff, coming from various disciplines, and with varying levels of enthusiasm for ACL. We were able to identify new barriers specific to scaling up the approach.

Refinement of observed barriers and proposed solutions

A project team was constituted at ARU with representation from all faculties, as well as a project steering group representing all areas of the University including senior management. These groups met regularly and provided input into solutions to the barriers.

We discussed our emerging themes at several conferences, both internal learning and teaching events, and to audiences from multiple institutions. We asked participants to comment on whether our findings reflected their own experiences of extending the use of ACL. Colleagues from UoB led a Twitter Chat on the subject of TBL, working with the ‘Learning and Teaching in HE’ Twitter community. The Twitter Chat was not designed for the identification of barriers per se, but inevitably some participants in the Twitter Chat raised problems they had encountered. We were then able to refine our suggested list of barriers incorporating new, albeit anecdotal, insights from colleagues.

Through this iterative process we continued to identify new barriers and proposed solutions to them. In the final phase of the project, we analysed further qualitative evidence from staff working on scaling up ACL, as well as outcome data, to determine whether the barriers had been successfully overcome.

Findings

Initial barriers

The barriers originally proposed for the business case for funding applications were:

• Fitting multiple summative assessments (required for TBL particularly) into existing university assessment regulations
• Changing the learning culture in some disciplines
• Moving from module level adoption to course level and strategic delivery
• The lack of a collaborative staff community sharing good practice and providing mutual support for developing ACL
• Practical issues relating to the learning environment such as availability of suitable rooms and resources

Multiple summative assessments

University assessment regulations are often based on traditional categories such as exams and coursework, which do not map well to the approach of continuous assessment or assessment for learning used in ACL (Sambell et al., 2013). Usually an exam is regarded as a single assessment element, while coursework is an assessment element consisting of multiple components. Regulations to avoid the risk of over-assessing students limit the number of assessment elements for a given module.

TBL, however, uses regular low-stakes summative tests (individual and team Readiness Assurance Tests – iRATs and tRATs) to engage students, support peer learning, and improve accountability (Michaelsen et al., 2002). iRATs and tRATs are exams in one sense, but regarding each as a standalone assessment would effectively preclude their being used summatively under most regulations. This is less of a problem for SCALE-UP, which does not traditionally use in-class summative tests (Beichner et al., 2007). Similarly, peer evaluation of group or team contributions can be seen as a separate assessment, which may not be permitted if multiple other assessments are already used.

Even where regulations are interpreted as allowing multiple tests, the practical problem of extracting a single combined mark which can be submitted into university assessment systems remains, especially if weighting or selection (e.g. best six out of eight tests) is applied. The traditional TBL approach, where tests are taken on paper and using ‘scratch cards’ (where participants reveal the correct or incorrect answer by rubbing off a coating), may require extensive manual transcription. Further, if the iRATs and tRATs are treated as ‘exams’ then students who have declared a disability may be entitled to reasonable adjustments (e.g. extra time, sitting the exam in a separate room, and so on). Such adjustments are sometimes impractical and frequently do not make sense in the TBL context. Likewise, regulations often require that all exams must be able to be retaken if a student misses the assessment date for a valid reason. However, it is not realistic to retake iRATs and tRATs as these are intended as learning tools, to develop accountability for team effort, and ensure students are ready to apply their new knowledge to solve problems (Michaelsen et al., 2002). Even if teams could be reconvened at a later date to retake a test, this would not be meaningful because the test was designed to prepare students for a class which had already taken place.

Conversely, if iRATs and tRATs are regarded as ‘coursework’ in that they continue throughout the module, they are not anonymous and as such would be subject to a process of blind double marking at some institutions, including ARU. Double marking is not usually feasible for rapid tests at the start of a class, and would not improve the reliability and grading fairness of a short set of multiple choice questions.

Because of this barrier, many academics have opted to use the iRAT and tRAT and/or the peer evaluation scores for formative purposes only. Yet, removing the summative element negates the theory behind TBL, where these regular summative tests ensure accountability for the completion of pre-session work.

There is some flexibility in the interpretation of regulations. One option we have explored is to include iRATs, tRATs, and peer evaluation marks under the heading of participation or engagement which can account for a proportion of a module mark under the regulations of universities in the project. Other modules accept treating the combined outcome of multiple individual tests as a single assessment element, as long as it can be demonstrated that this is not a way to circumvent the intention of the regulations to limit over-assessment. ARU is moving towards formalizing and providing infrastructure to support multiple components within a single assessment element.

To assist with the practicalities of combining multiple in-class test scores into a single reported mark, the project employed a Data Analyst at ARU, who worked with academics to identify examples of assessment practice, and to support and partially automate these processes. For the future, we are exploring possible software solutions for electronic approaches to TBL assessment.

Colleagues have been supportive in adapting regulations to match the reality of ACL. For example, disabled students’ official Statements of Reasonable Adjustments at ARU now specify end of module exams and note that in-class tests may have a different set-up. The regulations allow tutors to offer remediation for missed iRAT and tRAT tests, as an alternative to an improbable resit opportunity. It is common practice to record the marks only from the best of students’ iRAT and tRAT scores, (e.g. the best 8 out of 10), limiting the impact of a single missed test.

Learning culture

Some departments are resistant to adopting ACL methods, especially in areas where academics feel that a great deal of factual content needs to be covered. They can be attached to traditions such as extended didactic lectures (Gibbs, 1981). Some subjects which currently use very little active learning have a long distance to travel in transitioning to ACL. Additionally, courses which lead to a professional qualification may be perceived to be restricted in how much change they can make to their learning, teaching and assessment methods, even if professional bodies are often supportive in practice.

Occasionally cultural resistance arises in a department for the opposite reason: in disciplines where learning and teaching is already highly active and interactive, there may be a perception that there is little need for changing to a specific form of ACL. For example, students in Arts subjects may work together to produce an output in some medium, and critique each other’s work, or students in natural sciences may already take part in field-work in teams, or work in groups in laboratories. These activities are arguably a form of active learning, but may lack some of the benefits of more structured methods.

As the most effective way to overcome cultural resistance is to provide clear evidence of the benefits of ACL, we promoted ACL at several staff development and practice sharing events. As the project acquired and analysed large-scale data on the outcomes of TBL for student success, we are in a position to build a compelling case for further adoption.

We have encouraged flexibility in partial adoption of elements of ACL in the case of reluctance or practical difficulties with applying the method in its traditional form. For example, some disciplines remove the flipped classroom element, and maintain didactic teaching such as lectures in place of pre-session independent learning, where, in the case of TBL, lecture content may be tested using the iRAT and tRAT. Preliminary evidence from the project suggests that blended approaches to TBL do improve outcomes, though the benefits may be less substantial than the original approach.

Moving from module-level adoption to course level and strategic delivery

Institutional structures can sometimes hamper major changes in delivery of a particular subject (Freeman, 2012). It is usually possible, and indeed expected, for an individual module leader to innovate in learning and teaching in a module, but changing an entire degree course can be very challenging. Curriculum changes that affect assessment and delivery format require an often cumbersome formal approval process. In the era of Consumer and Marketing Authority (CMA, 2015) scrutiny of HE provision, advertised course descriptions need to match actual delivery and this requirement can further impede changes.

The key feature of a strategic implementation at course level is not the number of sessions or modules which use ACL, but whether the use of ACL forms an intentional part of curriculum design. Several academics have reported that there is simply no time for the whole course team to meet and coordinate adoption of new teaching methods. The ability to change the course-level curriculum depends very much on the culture within a particular department. Supportive senior management can provide a framework within which individual academics can collaborate. However, bottom-up change can only flourish with top-down support.

The capacity to integrate ACL both at course level and at institutional level is the area of greatest variation between the three institutions in the project. NTU has a large number of courses which include multiple SCALE-UP modules. UoB started using TBL within a particular course in Pharmacy, and much of their adoption of new modules has been at course level, but in fewer courses. ARU has increased the number of modules using TBL, and has some courses with several TBL modules, but there is so far slow progress towards embedding TBL in the course curriculum.

The creation of new courses, or the revalidation of existing ones, affords an opportunity to implement ACL. The newly created combined Health faculty at ARU has started several new courses as of 2018/19, which use TBL strategically as part of a blended delivery. To address issues around finding time for course teams, we have proposed updating the staff workload model so that changing the teaching approach to use ACL is allocated time on a similar basis to developing a new module. Additionally, we recommend the provision of ring-fenced time for course teams to meet regularly to review and address curriculum changes. We are engaging with course leader training to provide support with embedding TBL in curriculum design.

Collaborative staff community

This barrier is very much related to the above; as ACL is typically adopted by individuals rather than course teams, staff are often left working on their own. The workload of developing appropriate pre-session study guides, in-class tests, and application exercises can, therefore, be a challenge. Consistency and quality of materials would improve if staff had colleagues to peer-review them.

We are working to create resources and guides which can be accessed online, such as suggestions for student induction to ACL based on research findings regarding preparedness and expectations. We are creating a repository where colleagues can share resources such as successful application exercises, which can also be used in training across disciplines.

Scaling up to institutional level means that increasing numbers of academics require training and development, which requires additional resources. When relatively inexperienced ACL practitioners provide training to other colleagues, the training may not be of equivalent quality and consistency to that received from experts, and variations in delivery may result in mixed outcomes. Meanwhile, academics newly trained in ACL may not have access to a more experienced colleague who can guide and mentor them in their implementation. At the same time, scaling up delivery is positive because a critical mass of practitioners enables better peer support for individuals.

At the start of the project, academics felt they were working on ACL in isolation. Despite sharing common interests with colleagues, they often did not know who else was using a similar approach across the University. They found it difficult to collaborate with colleagues who were not in the same building or even on the same campus, not to mention the challenges involved in working across disciplines.

The project has resulted in institutional knowledge of who is using ACL, which has facilitated connections between colleagues working in related disciplines, countering the sense of isolation. At ARU the project identified TBL leads in each faculty, who are helping to establish links between practitioners and form a meaningful Community of Practice (Wenger, 1998). One approach to sustaining ACL beyond the lifetime of the project could be to form a User Group at university level, perhaps initially based on the network of faculty TBL leads.

In contrast to the US where methods such as TBL have a high profile, ACL networks are relatively weak in the UK and Europe, and there are few practitioners in Europe (although this number is increasing). UoB and ARU have contributed to the formation of the European TBL Community, supporting the use of TBL in Europe, and contribute to national active learning networks linking universities that use ACL. Project institutions have also hosted national ACL conferences.

Practical issues

The SCALE-UP approach to ensure effective group working and dialogue between different groups is based on a particular arrangement of furniture, large round tables permitting groups of three to merge into groups of nine. Reporting group outputs also expects availability of a shared laptop for each group (Beichner et al., 2007). However, although TBL is relatively agnostic about learning environment, technology, and room layout, it is less well suited to traditional lecture theatres with seating in fixed, banked rows (Yuretich and Kanner, 2015), yet these are the most common arrangement for large teaching spaces in most universities.

Where specialist rooms with furniture designed for group collaboration are available, they are often in high demand. IT facilities designed to support ACL are attractive to many academics even if they are not using the specific methods the equipment was installed for. Timetabling has a long lead-in time for academics to book a small number of specialist active learning rooms. Staff may not decide to adopt TBL until after the deadline for booking rooms for teaching in the upcoming period.

Increasing student numbers also put increased pressure on limited physical space. This problem is particularly acute in HEIs where the campus space cannot be expanded with new builds. Tables arranged in rows allow more students to be accommodated than tables arranged in the café-style, which is more conducive to group work. It is not always feasible, or permitted, to move furniture between teaching sessions.

Timetabling and Estates departments have been supportive of quick-win solutions such as provisioning more rooms set up in café-style rather than rows by default. In the longer term, more learning spaces can be adapted for active learning in line with the regular upgrade and refurbishment cycles.

All three partners have benefited from investment in creating new active learning rooms, both through new build and converting existing spaces. New lecture theatres have been furnished with adaptations for ACL, such as swivel seats, allowing students to collaborate across rows. Improving and creating learning spaces to promote active learning is a current priority for much of the sector, so eventually more suitable spaces for ACL will become available.

Emerging themes

As ACL is extended beyond enthusiastic early adopters, some challenges become more prominent. For example, the ARU pilot in 2015 reported only a few student complaints that ACL did not match how they expected to be taught (perhaps based on their experience in school and their limited exposure to student-centred learning approaches), or did not provide value for money. As ACL is scaled up from early adopters, it may be delivered by less enthusiastic academics, which can have a negative impact on student satisfaction. We are addressing this through staff development to improve student induction to ACL methods, as well as design of activities to promote a positive team dynamic.

Another factor related to scaling up is that with more students experiencing ACL, there is a higher chance of encountering a particular student or group of students for whom the approach may be less suitable. There is currently little information available about adapting ACL to include students with social disabilities such as autism or social anxiety, or those with sensory impairments, such as hearing impaired students, for whom the noisy environment of multiple overlapping group discussions can present a significant barrier.

Anecdotally, it seems that early adopters found ways to adapt their approach for individual students in their class with particular learning needs. The broader pool of practitioners now using ACL may be less well equipped to make these adjustments. There is no record available of good practice in the area of adapting ACL to meet learning needs (Kent et al., 2015), and academics do not have networks of colleagues experienced in the methods. With traditional approaches such as lectures, where there is far more literature and guidance available, appropriate adjustments can more easily be made. As ACL is more widely adopted there will be opportunities to gather research evidence and create guidance for adapting ACL to support all students. Table 7.1 shows barriers and solutions we have identified so far.

Conclusion

Embedding educational change on an institutional scale requires support from all levels of the institution. The different experiences of the three project partners have highlighted the importance of collaboration between senior management, mid-level leaders such as heads of department, and staff implementing the teaching innovations. Collaboration between all areas of the universities, including Academic Registry, Estates, Timetabling and other professional services, has been key to project success.

Although regulatory barriers featured prominently in the initial analysis, these barriers can largely be resolved with goodwill from stakeholders. There is a widespread understanding that regulations should support rather than undermine the student experience. There are always practical difficulties surrounding learning spaces and budget for equipment, but our experience matches reports in the literature showing that ACL is not dependent on particular technology or room layouts, but can be delivered effectively in any space (Michaelsen et al., 2002; Beichner et al., 2007).

A more significant barrier is building a Community of Practice around TBL. Moving from individual to course level adoption has proved very challenging, even with resources put in place by the project to support this outcome. This can affect student experience where course delivery is not consistent. Importantly, where individual staff adopt ACL in isolation, there are knock-on effects on staff workload, because academics have to prepare all materials on their own, whereas they could be sharing, reviewing, and improving each other’s resources.

Sustainable scaling up also relies on cross-institutional collaboration, and leadership at national level. The project has demonstrated the value of three institutions working together. Despite very different institutional contexts, the partners have been in a position to share good practice. Collaboration has promoted finding imaginative solutions to barriers shared across the project institutions.