National Institutes of Health Stroke Scale Certification Is Reliable Across Multiple Venues

NIH Stroke Scale certification is required for participation in modern stroke clinical trials and as part of good clinical care in stroke centers. A new training and demonstration DVD was produced to replace existing training and certification videotapes. Previously, this DVD, with 18 patients representing all possible scores on 15 scale items, was shown to be reliable among expert users. The DVD is now the standard for NIH stroke scale training but the videos have not been validated among general (i.e. non-expert) users. We sought to measure inter-rater reliability of the certification DVD among general users using methodology previously published for the DVD. All raters who used the DVD certification through the American Heart Association website were included in this study. Each rater evaluated one of 3 certification groups. Responses were received from 8214 raters overall, 7419 raters using the internet and 795 raters using other venues. Among raters from other venues, 33% of all responses came from registered nurses, 23% from Emergency Department MD/Other ED/other physicians, and 44% from neurologists. One half (51%) of raters were previously NIHSS certified and 93% were from United States/Canada. Item responses were tabulated, scoring performed as previously published, and agreement measured with unweighted kappa coefficients for individual items and an intraclass correlation coefficient for the overall score. In addition, agreement in this study was compared to the agreement obtained in the original DVD validation study to determine if there were differences between novice and experienced users. Kappa's ranged from 0.15 (Ataxia) to 0.81 (LOC-C questions). Of 15 items, 2 showed poor, 11 moderate, and 2 excellent agreement, based on Kappa scores. Agreement was slightly lower to that obtained from expert users for LOCC, Best Gaze, Visual Fields, Facial Weakness, Motor Left Arm, Motor Right Arm and Sensory Loss. The intraclass correlation coefficient for total score was 0.85 (95% CI 0.72, 0.90). Reliability scores were similar among specialists and there were no major differences between nurses and physicians, though scores tended to be lower for neurologists and trended higher among raters not previously certified. Scores were similar across various certification settings. The data suggest that certification using the NINDS DVDs is robust and surprisingly reliable for NIHSS certification across multiple venues.