Establishing Construct Validity: The Cases of Risk Preference and Exploratory Factor Analysis

A crucial precondition for being able to test scientific theories is to clearly define relevant constructs and to validate their assessments. The process of construct validation has been divided into six aspects that focus on different domains of validity evidence, ranging from theoretical considerations to the consequences of assessments and respective score interpretations. In the four manuscripts presented in this dissertation, I focused on several aspects of construct validation in measures of risk preference, as well as on a particular method to investigate the structural aspect of construct validity. Specifically, in manuscript one we investigated the content and substantive aspects of construct validity of self-reported risk preference by focusing on people’s cognitive representations of their risk preferences, as well as on potential information integration processes involved during judgment formation. Our results provide further evidence for the validity of assessing risk preference using self reports. In manuscript two, we focused on a different approach to assessing risk preference: behavioral tasks. Specifically, we investigated and aimed to improve the content, substantive, and external aspects of construct validity of the balloon analogue risk task (BART). Adapting the stochastic structure of the BART by following the principles of representative design, we were able to improve the task’s content and substantive validity aspects, but not its external validity aspect. Manuscript three presents the EFAtools R package that we created to facilitate (a) the process of structural validation of operationalizations, and (b) the comparison of the implementations of a popular exploratory factor analysis (EFA) procedure in R and SPSS. In manuscript four, we then used this package to investigate why this EFA procedure produces differing results when conducted in R than when conducted in SPSS, and whether one of the two implementations should be preferred in construct validation. We found a total of five differences between the two implementations of the EFA procedure that sometimes led to substantial differences in the obtained structural validity evidence. Moreover, we were able to identify an implementation that, on average, maximizes the structural validity evidence obtained with the investigated EFA procedure. With these four manuscripts, this dissertation provides a small, incremental step in the direction of valid assessments of the construct of risk preference, and of improving one of the tools often employed to establish structural validity evidence.