Using Survival Analysis to Understand Patterns of Sustainment within a System-Driven Implementation of Multiple Evidence-Based Practices for Children’s Mental Health Services

Evidence-based practice implementation requires substantial resources in workforce training; yet, failure to achieve long-term sustainment can result in poor return on investment. There is limited research on EBP sustainment in mental health services long after implementation. This study examined therapists’ continued versus discontinued practice delivery based on administrative claims for reimbursement for six EBPs (Cognitive Behavioral Interventions for Trauma in Schools [CBITS], Child-Parent Psychotherapy [CPP], Managing and Adapting Practices [MAP], Seeking Safety [SS], Trauma Focused Cognitive Behavior Therapy [TF-CBT], and Positive Parenting Program [Triple P]) adopted in a system-driven implementation effort in public mental health services for children. Our goal was to identify agency and therapist factors associated with sustained EBP delivery. Survival analysis (i.e., Kaplan-Meier survival functions, log-rank tests, Cox regressions) was used to analyze 19 fiscal quarters (i.e., approximately 57 months) of claims data from the Prevention and Early Intervention Transformation within the Los Angeles County Department of Mental Health. These data comprised 2,322,389 claims made by 6873 therapists across 88 agencies. Survival time was represented by the time elapsed from therapists’ first to final claims for each practice and for any of the six evidence-based practices. Results indicate that therapists continued to deliver at least one evidence-based practice for a mean survival time of 21.73 months (median = 18.70). When compared to a survival curve of the five other evidence-based practices, CBITS, SS, and TP demonstrated higher risk of delivery discontinuation, whereas MAP and TF-CBT demonstrated lower risk of delivery discontinuation. A multivariate Cox regression model revealed that agency (centralization, service setting) and therapist (demographics, discipline, case-mix characteristics) characteristics were significantly associated with risk of delivery discontinuation for any of the six evidence-based practices. This study illustrates a novel application of survival analysis to administrative claims data in system-driven implementation of multiple evidence-based practices. Findings revealed variability in long-term continuation of therapist-level delivery of evidence-based practices and highlight the importance of both agency and workforce characteristics in the sustained delivery of EBPs. Findings direct the field to potential targets of sustainment interventions (e.g., strategic assignment of therapists to EBP training; strategic selection of evidence-based practices by agencies).