Applying a governance barometer to vaccine delivery systems: Lessons from a rural district of Pakistan

Abstract Introduction Weak vaccine delivery systems in countries off-track for routine immunization targets, need in-depth evidence on system level barriers to be critically resourced and monitored. We applied a Balanced Score Card (BSC) approach in a rural underserved district of Pakistan to (i) identify critical areas needing support in the government vaccine delivery system; and (ii) for benchmarking improvements in the vaccine delivery system. Methodology BSC was developed drawing on desk review, government consultations and field testing. 45 immunization indicators were finalized across 8 domains: human resource; vaccine supply; safe vaccination practice; cold chain maintenance; outreach preparedness; records & supervision; verifiable vaccination volume; and client communication. Data were collected through health facility assessments, client exit interviews and household vaccination assessment. A composite score was calculated for each domain and banded into unsatisfactory, borderline and satisfactory categories. 5 lowest ranking domains were targeted for 2 years of health systems strengthening (HSS) interventions. Post-intervention assessment tracked progress. Results The district obtained a cumulative score of 51% (unsatisfactory) at pre-intervention and improved to 82% (satisfactory) at post-intervention. At pre-intervention, 4 domains scored satisfactory and 4 scored unsatisfactory. Unsatisfactory scores were received for: outreach preparedness; records & supervision; verifiable vaccination volume; and client communication. Post intervention 6 of 8 domains scored satisfactory and 2 moved from unsatisfactory to borderline. Highest percentage point (pp) improvements were seen in outreach preparedness (53 pp, p = 0.01), EPI supervision (52 pp, p = 0.01) and verified vaccination volume (46 pp, p = 0.02). 3 domains that were not intervened through HSS interventions had minimal change in scoring - cold chain maintenance (6 pp), safe vaccination practice (12 pp) and vaccine supply (11 pp). Conclusion BSC served to prioritize interventions towards critical unmet needs for vaccine delivery in the district health system and particularly helped to improve outreach preparedness, EPI supervision and verified vaccination volume.