IHI ID 22 Validation of the quality improvement assessment (QIA) tool

Background QI training occurs regularly in health care systems often without impact assessment. QIKAT is scenario-based, is focused on confidence not skill attainment or organizational and team contexts. A validated assessment tool grounded in QI theory, capturing organizational context, skills and growth, for frontline healthcare members is needed. The 67 item scale QIA survey used at Dartmouth measures QI skills, communication, team dynamics, and context. Response options are ordered categories measured on a 5-point scale. QIA data from 463 individuals in nine programs from 2016–2017 were used in this study. Objectives Establish validity and reliability of Quality Improvement Assessment Survey. Methods Content validity: Literature review/interviews Construct validity: Exploratory/confirmatory factor analysis Redundant/poor performing items: Item analysis/Rasch modeling. Results Literature review and consultation confirmed inclusion of measures of QI skills, organizational culture, and team dynamics. EFA of QI skills items loaded on four factors measuring System, Variation, Psychology and Knowledge. EFA on organizational culture and team dynamics yielded four factors measuring Microsystem and Organizational leader(s) and Awareness of Self and Others in Team Dynamics (table 1). CFA confirmed data fit the hypothesized theoretical model for QI skills (CFI=0.954, RMSEA=0.075) Organizational culture/team dynamics (CFI=0.969, RMSEA=0.049) (table 2). Scales were reliable and moderately correlated (see table 3). Item analysis/Rasch modeling identified several items that were redundant or poor performing. Conclusions Study provides evidence QIA survey possesses strong psychometric properties. High model fit indices show the data reflect theoretical models of organizational change. Moderate correlations between scales indicate assessment of distinct constructs and high scale reliabilities signify precise and robust measurement.