Assessing the Impact of Writing Development through Consensus - Comparing Subject Lecturers and Learning Developers

Identifying the impact of writing development is an important part of developing an evidence-based approach to learning development. Developing an evidence-base for writing development cannot be achieved without measurement tools that are appropriate for the complex notion of writing as social interaction, which underlies an academic literacies approach to teaching writing. Previous conceptual work (Canton. In press) suggests that adapting Amabile’s Consensual Assessment Technique (CAT) for writing has the potential to develop a tool that evaluates writing in terms of successful communication, but delivers reliable, quantitative results. An empirical study that uses a newly-developed measurement tool based on consensus is currently exploring the potential of evaluating writing through the CAT.
The proposed project explores the conditions under which the new tool can be used further. Since the tool takes into consideration the situated nature of writing, familiarity with the specific communicative context in which a text was produced is an essential requirement for raters. The proposed study therefore aims at testing whether consensus on the degree to which texts successfully communicate can be achieved within and between two different groups: learning developers and subject lecturers. Widening the group of potential raters would contribute significantly to developing a measurement tool that allows learning developers to assess writing, and potentially the impact of their teaching, in a way that is appropriate for their conceptual understanding of writing and pedagogical framework for teaching, but can deliver reliable quantitative results.