The whole is more than the sum of its parts – assessing writing using the consensual assessment technique

Daniela Zahn, Ursula Canton*, Victoria Boyd, Laura Hamilton, Josianne Mamo, Jane McKay, Linda Proudfoot, Dickson Telfer, Kim Williams, Colin Wilson

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Downloads (Pure)

Abstract

Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. “Student Writing in Higher Education: An Academic Literacies Approach.” Studies in Higher Education 23 (2): 157–72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers (1) gain better understanding of what influences written social interactions, and (2) improve their ability to manipulate language to address readers. Self-report can assess the first; the second can only be evaluated by examining texts and their effect on readers. Texts are commonly assessed with rubrics-based tools focussing on textual features, but their insensitivity to communicative context and readers’ perception makes these inappropriate for an Academic Literacies framework (Canton [2018. “‘It’s Hard to Define Good Writing, but I Recognise it When I See it’: Can Consensus-Based Assessment Evaluate the Teaching of Writing?” Journal of Academic Writing 8 (1): 13–27. doi:10.18552/joaw.v8i1.450]). Consensual assessment used by Amabile ([1996. Creativity in Context. Boulder, CL: Westview Press]) for creativity, offers a potential solution (Canton [2018. “‘It’s Hard to Define Good Writing, but I Recognise it When I See it’: Can Consensus-Based Assessment Evaluate the Teaching of Writing?” Journal of Academic Writing 8 (1): 13–27. doi:10.18552/joaw.v8i1.450]). This paper presents a new instrument based on consensual assessment and empirically tests it. Intra-class Correlation Coefficients (ICCs) found moderate (Koo and Li [2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15 (2): 155–63. doi:10.1016/j.jcm.2016.02.012]) to good (Cicchetti [1994. “Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology.” Psychological Assessment 6 (4): 284–90. https://pdfs.semanticscholar.org/50d7/f68422d0c0424674f6b235ac23be8300da38.pdf]) agreement among raters, which offers proof of concept for capturing the readers’ perception of the complex interactions in writing.
Original languageEnglish
Pages (from-to)1907-1918
Number of pages13
JournalStudies in Higher Education
Volume46
Issue number9
Early online date24 Jan 2020
DOIs
Publication statusPublished - 2 Sep 2021

Keywords

  • consensus
  • writing
  • academic literacies
  • assessment
  • ICC

Fingerprint

Dive into the research topics of 'The whole is more than the sum of its parts – assessing writing using the consensual assessment technique'. Together they form a unique fingerprint.

Cite this