Adapting a Small Group Communication Quality Assessment to New Contexts

Objectives : Small group learning is a well-established medical education strategy for cultivating essential communication skills. Yet, how best to measure communication quality in these groups remains understudied. The objective of this study was to adapt a communication methodology (Communication Quality Analysis) to medical education small group setting. Methods : This was an observational study of Preclinical Medical Humanities small group discussions. Groups were recruited by convenience sampling. Audio-recordings of 12 sessions (3 groups; n=22 students and 3 facilitators) were transcribed and analyzed using Communication Quality Analysis. Three coders assessed communication quality by assigning numeric scores based on how well participants accomplished communication goals within five domains: content, engagement, relationship, emotion and identity. Dialogue was coded every five minutes for each domain, resulting in 2,658 data points for analysis. Coder reliability was assessed using intra-class correlations. Variance components were assessed using a generalized linear model. Results : CQA was successfully adapted to the small group education context. High inter-rater reliability was established for each of five communication quality domains (ICC range 0.875 to 0.98). Variability in scores the relationship and identity scores was based primarily on the duration of class (ie. 5 minutes into class versus 35 minutes into class). Variability in the content, emotion, and engagement scores was based primarily on the the participant (who was speaking). Considerable variability in domain scores was observed between participants, suggesting that the assessment is sensitive enough to detect nuanced differences between participants. Conclusions : Our study shows that CQA is reliable when adapted to medical education small groups.