Active assessment literacy

What is the idea?

A common challenge across the higher education (HE) sector relates to assessment, especially students’ understanding of the specifics around assessed activity. Drawing on interdisciplinary case studies, this chapter explores the role of collaborative analysis and annotation practices in helping students to develop assessment literacies. In doing so, the chapter demonstrates that collaborative annotation can be a powerful tool for improving students’ knowledge and understanding of assessment requirements, extending across past submission analysis, research proposals, and ethics submissions. Talis Elevate, a collaborative annotation tool, is the platform used to facilitate assessment literacy activities in this example, but the idea can be applied using various technologies.

Why this idea?

Assessment literacy is critical for several reasons. Assessments are integral to HE since they are often used to assure academic and professional standards, measure whether learning outcomes have been achieved, and determine students’ degree classifications (McConlogue, 2020). Assessments are also tools for learning that help students to identify their strengths and areas for development, promoting critical reflection that will serve them well during and beyond university (Carless & Boud, 2018; Sambell, McDowell, et al., 2012). Yet many students struggle to understand assessment criteria or why assessments matter for their learning beyond the attainment of grades, creating dissatisfaction with assessments and causing frustration when expectations and desired outcomes are not achieved (Sambell, Brown, et al., 2017; Winstone & Boud, 2021). At the same time, educators might assume students understand assessment criteria and miss opportunities to embed assessment literacies into course design, impacting students who are unfamiliar with assessment activities, processes, and expectations.

Collaborative annotation offers many pedagogical potentials regarding academic skills development. Cohn (2020) addresses several educational opportunities that arise from collaborative digital reading and annotation activities, such as developing shared understandings of the syllabus via annotation. Chen et al. (2010) explored collaborative annotation tools to correct misunderstandings of assessment activity, create shared understandings of assessment criteria, and identify writing errors, demonstrating impact not only on students’ assessment literacy but also general reading comprehension.

Despite these opportunities, the possibility for collaborative annotation to build assessment literacies is not widely researched. In what follows, we outline an activity with step-by-step instructions you can use to develop students’ assessment literacies. The activity is intended to create meaningful learning experiences through which students are empowered to reach their full potential in assessments.

How could others implement this idea?

Step 1: Introduce students to the assessment and discuss the criteria with them, ensuring opportunities for questions to identify and clarify any areas of uncertainty. Following constructive alignment principles, this type of activity works well early on in a module to allow students time to practise and apply what they learn to their own assessments (Biggs & Tang, 2011).

Step 2: Provide students with a variety of assessments, either ‘mock’ submissions or prior student submissions (with permissions) across grade boundaries for them to peer review, using a collaborative annotation tool such as Talis Elevate. These exemplars could illustrate good, satisfactory, and poor practice, but can also focus on other activities such as interpretation of writing styles, identifying plagiarism and collusion, or highlighting elements you feel will spark discussion and debate among the learning community. We recommend excluding grading from this process to ensure feedback is the core goal of the task and reduce the focus on grades (Barnes, 2018).

Step 3: Outline the collaborative peer review activity, including why it is beneficial for learning development, how long the task should take, who students will work with during the activity, and what tool they will use to complete it. You need to ensure students can access and use the collaborative annotation tool, which you can ascertain by providing a low- or no-stakes activity (e.g. instructing students to make one ungraded annotation on a source to familiarise them with the annotation tool itself). If collaborative annotation activities are new for students, it is equally important to frame the expected social discourse within such an educational domain (Cohn, 2021).

Step 4: Assign students into small groups and explain that you want them to work together to provide constructive feedback on the exemplars, using the assessment criteria to help them identify strengths and areas for development. Establish ground rules for respectful interaction, i.e. explain that when they disagree with a peer’s annotation, they should support their disagreement with critical rationale and evidence, before engaging in discussion. The aim is to come to a consensus, mimicking common peer review processes used in academia and workplace reviews.

This feedback should be added to the annotations at relevant points in the exemplars on the collaborative annotation tool so comments are visible to the wider group. Where consensus is reached, it should be made explicit to the learning community to help them identify good practice to take forward in their own assessments. Encouraging criticality, whilst maintaining respect for the author and providing constructive feedback, is essential for developing students’ assessment literacies in a supportive environment.

Step 5: Allocate time after the small group activity to discuss the annotations as a whole group. This discussion activity creates opportunities not only to talk through (mis)understandings about the assessment and peer review activity, but also encourages students to reflect on what they learnt in relation to their own academic practice. For example, you can encourage students to consider their own strengths and areas for development for them to address in preparation for their own assessment.

While ‘active assessment literacy’ can be adopted for synchronous, asynchronous, online, or in-situ modalities, we recommend a process of reflective review is undertaken in a synchronous space to allow for deeper discussion and collective synthesis.

Transferability to different contexts

Challenges around assessment literacy are ubiquitous across disciplines and modes of study. The above collaborative annotation activity has proven useful for addressing these challenges across various HE contexts and assessment formats. For example, it has been used by students to peer review previous assignment submissions in English (Merrydew, 2022), ethics applications in Sport and Exercise Science (East, 2019), and research proposals in Social Sciences, demonstrating its transferability and usefulness as a tool for active assessment literacy development.

Links to tools and resources

• Talis Elevate website: https://talis.com/talis-elevate/
• Talis Elevate blog: https://talis.com/category/talis-elevate/

References

Barnes, M. (2018, January 10). No, students don’t need grades. Education Week. https://www.edweek.org/teaching-learning/opinion-no-students-dont-need-grades/2018/01

Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: What the student does (4th ed.). Open University Press.

Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Cohn, J. (2021). Skim, dive, surface: Teaching digital reading. West Virginia University Press.

Chen, J., Chen, M. C. & Sun, Y. S. (2010). A novel approach for enhancing student reading comprehension and assisting teacher assessment of literacy. Computers and Education, 55(3), 1367–1382. https://doi.org/10.1016/j.compedu.2010.06.011

McConlogue, T. (2020). Assessment and feedback in higher education: A guide for teachers. UCL Press. https://discovery.ucl.ac.uk/id/eprint/10096352/

East, M. (2019, July 24). Using Talis Elevate for collaborative annotation of ethics submissions: Hear from a principal lecturer at Anglia Ruskin University. Talis. https://talis.com/2019/07/24/talis-elevate-case-study-adrian-scruton-principal-lecturer-and-academic-lead-employability-anglia-ruskin-university/

Merrydew, A., (2022, February 14). Confidence is key: Building students’ academic reading literacies through collaborative annotation, Making Digital History. https://makingdigitalhistory.co.uk/2022/02/14/confidence-is-key-building-students-academic-reading-literacies-through-collaborative-annotation/

Sambell, K., McDowell, L., & Montgomery, C. (2012). Assessment for learning in higher education (1st ed.). Routledge.

Sambell, K., Brown, S., & Race, P. (2017). Helping students appreciate what’s expected of them in assessment. Edinburgh Napier University. https://lta.hw.ac.uk/wp-content/uploads/GUIDE-NO8_Assessment-Literacy.pdf

Winstone, N. E. & Boud, D. (2020). The need to disentangle assessment and feedback in higher education. Studies in Higher Education, 1–12. https://doi.org/10.1080/03075079.2020.1779687