A Model Selection Approach to Testing Dependent ICCs

In this commentary, we discuss the tutorial by Cohen and Doveh (this volume) on using multi-level models to estimate and test hypotheses about (dependent) intra-class correlations (ICCs). The goal in this context is to find subsets of homogeneous ICCs across time points and/or experimental tasks. We suggest that one should approach this model selection problem by (a) specifying the set of all possible models, (b) using a systematic top-down selection strategy that avoids possible inconsistent and intransitive patterns in the results, and (c) using the entire dataset available when fitting each model. Our goal is to discuss the larger framework in which the specific models considered by Cohen and Doveh in their analysis are embedded, which should be useful in guiding researchers faced with a similar analysis task.