Rating Scale Format and Item Sensitivity to Response Style in Large-Scale Assessments

This study examines the relationship between rating scale format and instrument sensitivity to response style (RS). A multidimensional nominal response model (MNRM) that allows item discrimination to vary across items for both substantive and RS dimensions can be used for this purpose. We first conduct a set of simulations to examine the recovery of item slopes under the model. Then we apply the model to PISA and PIRLS survey items to investigate the item sensitivity to RS in relation to different rating scale types. Simulation results indicate good recovery of item slopes on both substantive and RS traits, and different rating scale formats (agreement-type versus frequency-type scales) from PISA and PIRLS are found to have varying sensitivities to RS, such that frequency-type scales are less affected.