Medication Administration Evaluation and Feedback Tool: Inter-rater reliability in the clinical setting

ABSTRACT Aims This study assessed the inter-rater reliability, acceptability and usability of the Medication Administration Evaluation and Feedback Tool for nurses in the clinical setting. Background Medication administration is a complex nursing task requiring multiple steps to ensure safe and accurate delivery of medications to patients. Currently, registered nurses are not routinely provided the opportunity for regular review of their practice. The Medication Administration Evaluation and Feedback Tool has been previously validated in the simulated environment. Methods Four nurse observers were trained to use the tool. Thirty nurses participated to be observed in the clinical setting. Each nurse was assessed simultaneously by two observers. Inter-rater reliability was assessed using Fleiss’ Kappa coefficient. A post-observation survey was conducted to assess user acceptability. The Guideline for Reporting Reliability and Agreement Studies Enhancing the Quality and Transparency of Health Research was used. Results The observed agreement between observers using the Medication Administration Evaluation and Feedback Tool in clinical practice was 0.90 and Fleiss’ kappa coefficient was 0.77 demonstrating excellent agreement and inter-rater reliability. Both nurses and observers reported the tool was useful and practical for use in evaluating medication administration practice in the clinical environment. Conclusions Inter-rater reliability testing of the Medication Administration Evaluation and Feedback Tool in the clinical environment demonstrated it is a reliable and valid tool when used by different observers. Both nurses and observers found using the tool a positive and useful experience when evaluating medication administration practice.