Model selection in observational media effects research: a systematic review and validation of effects

Media effects research has produced mixed findings about the size and direction of the relationship between media consumption and public attitudes. We investigate the extent to which model choices contribute to these inconsistent findings. Taking a comparative approach, we first review the use of different models in contem- porary studies and their main findings. In order to extend and validate this review, we consider the implications for national election studies attempting to measure media effects in election campaigns and recreate these models with the British Election Study 2005–2010 panel data. We compare the direction and size of effects of media content on attitude change across: between- subjects, within-elections models, in which the effects of indivi- dual-level variance in media exposure and content are assessed; within-subjects, within-elections models, which compare the effects of variance in media content for the same individual; and within-subjects, between-elections models that allow us to analyse the links between media content and exposure with attitude change over time. Our review shows some notable differences between models in terms of significance of effects (but not effect sizes). We corroborate this finding in the British campaign analysis. We conclude that to check the robustness of claims of media effects in observational data, where possible researchers should examine different model choices when evaluating media effects.