Measurement Reactivity in a Randomized Clinical Trial Using Self‐Reported Data

Self-reported measures, such as diaries, checklists, interviews, or surveys, are common in behavioral interventions. In randomized clinical trials (RCTs), the use of these measures can lead to internal validity concerns due to the testing effects (i.e., Hawthorne effect or measurement reactivity) that these measures may elicit. McCambridge, Witton, andElbourne (2014) operationalized the Hawthorne effect by describing it as the testing effect that occurs when participants are assessed by being directly observed, made aware of being studied, or asked to report on their behavior by answering questions. Measurement reactivity is a special case of the Hawthorne effect that specially focuses on the processes by which repeated measurement leads to changes in the participants being measured (French & Sutton, 2010; Shadish, Cook, & Campbell, 2002). Specifically, the measures themselves may inform the study participants of the intended behaviors being evaluated and in turn influence the achieved results. This can create questions regarding whether or not the intervention was effective (Gravetter & Forzano, 2009).In a 1-year randomized trial teaching self-management skills to 202 people with long-term indwelling urinary catheters, both groups appeared to have improved. The research team attributed this to an increase in selfmonitoring (awareness and observations or recordings) caused by use of the catheter calendar, but this conclusion was not supported by data from the study. The improvements of decreasing symptomatic catheter-related urinary tract infection and catheter blockage over the 12 month study were clinically meaningful in both the control and intervention groups (Wilde et al., 2015). Importantly, similar improvements were not observed in a prospective observational study of 34 catheter users of the same population who were assessed bimonthly over a 6 month period (Wilde et al., 2010), causing the researchers to wonder what contributed to both groups' improvements in the 12 month RCT.The unintended improvement in the control group compelled the researchers to try to determine its potential cause, specifically whether the calendar or other factors caused behavior changes in the participants. We believed that the catheter calendar, used to record catheter problems, could have contributed to selfmonitoring and in turn measurement reactivity in the control group. The answers to the final open-ended interview question for both the experimental and control groups were analyzed. The question asked was,Now that the study is over, what can you tell us about how it went? Please tell us whether it was worth it to you to be in the study and why? (Note to interviewers: Describe in participant's words as much as able.)This is a discussion of the data analyzed from the responses to this question, the measurement issues in selfreported data, implications related to self-monitoring, and suggestions for minimizing this testing effect in repeated measures studies, particularly self-management RCTs.BackgroundSelf-management clinical trials often use self-reported data for health status, health indicators, and healthcare utilization outcomes. Several recent Cochrane reports on self-management show low to moderate benefit in outcomes for conditions like osteoarthritis (Kroon et al., 2014), asthma (Peytremann-Bridevaux, Arditi, Gex, Bridevaux, & Burnand, 2015), and type 2 diabetes (Pal et al., 2013). In a meta-analysis of outcomes related to behavior, physical and psychological health, and healthcare utilization, based on the Stanford Chronic Disease Self-Management Program (CDSMP) literature from 1999 to 2009, researchers suggested small to moderate effects in studies measured at baseline and at 6 and 12 months (Brady et al., 2013). In this review of the 23 studies, 9 were RCTs and the rest were longitudinal pre- and post-research. The CDSMP has been tested in multiple studies, including clinical trials for specific conditions like arthritis using peer leaders (Lorig, Ritter, Laurent, & Fries, 2004), for varying chronic conditions in online programs (Lorig et al. …