Increase of reliability by incorporating response time into the paired-comparison psychological measurement

Much scholarly attention is focused on psychological measurement through the paired-comparison format, which is considered to be tolerant to systematic response bias. The Thurstonian D-diffusion item response theory model was recently proposed to incorporate response-time information in this context. Because reliability is a fundamental measurement property, this study used the above model to conduct a preliminary investigation into the extent of the reliability increase achieved when incorporating response-time information into paired-comparison psychological measurement. Under some realistic conditions, our simulation revealed a practically relevant (but not very large) degree of increase. The same type of increase was also found during our analysis of a real psychological dataset containing measurements for the Big Five traits. As such, this study produced evidence supporting the collection and utilization of response time when conducting paired-comparison psychological measurement.