Effect of short training on inter- and intraobserver agreement for the assessment of dyspnoeic children

The aim of the present study was to evaluate the effect of a short dyspnoea observation training of observers on their inter- and intraobserver agreement in the assessment of dyspnoeic children. Thirty video recordings of dyspnoeic children were independently assessed by four experienced paediatricians, twice before training and twice afterwards, with 1-wk intervals. Assessments included wheezing, prolonged expiration, air entry, retractions (subcostal, intercostal, jugular and supraclavicular), nasal flaring, ability to speak, mental status and an overall dyspnoea score on a 10-point Likert-scale. The training was a 30 minute meeting, in which paediatricians discussed their assessment of five video recordings of dyspnoeic children. Agreement was assessed by multirater kappa and raw agreement for binary variables, and with intraclass correlation coefficient (ICC) for the overall dyspnoea score. We analysed 478 assessments. Intraobserver agreement of clinical findings was moderate to substantial (kappa 0.40-0.85, raw agreement 75-95%). Interobserver agreement was slight to fair, (kappa 0.10-0.40, raw agreement 60-80%). The overall dyspnoea score showed good intraobserver (ICC 0.76) and moderate interobserver agreement (ICC 0.62). Training did not improve agreement on individual clinical findings (p>0.05), but did for the overall dyspnoea score (ICC 0.86 and 0.68 for intra- and interobserver agreement after training, respectively). Results suggest that an overall impression of dyspnoea is less variable between clinicians than individual clinical findings associated with dyspnoea.