3.15 Measurement properties of the adult version of the sport concussion assessment tool 5th edition symptom evaluation using Rasch analysis
0
Citation
0
Reference
10
Related Paper
Abstract:
Objective
To evaluate the psychometric and measurement properties of the 5th edition of the Sport Concussion Assessment Tool (SCAT5) Symptom Evaluation using Rasch analysis.Design
Cross sectional study using Rasch analysis.Setting
Single Site – Primary Care Setting.Participants
A total of 284 participants who were still experiencing concussion symptoms were included (130 males, 154 females, mean age 20.8 ±10.4). Participants were 13 years of age or older, with a diagnosis of concussion from a primary care physician.Interventions
The SCAT5 symptom evaluation was administered to patients as a component of their routine clinical encounter and the presence and severity of each of the 22 symptoms was included in the analysis.Outcome Measures
Rasch analysis was performed using RUMM 2030 to assess the SCAT5 symptom evaluation for overall fit, response scaling, individual item fit, differential item functioning, local dependency, unidimensionality and reliability.Main Results
The SCAT5 symptom evaluation demonstrated an acceptable fit to the Rasch model, exhibited high reliability and was able to differentiate between at least 4 levels of patients. Nonetheless, serious psychometric issues were identified. Response dependencies were identified between 15 pairs of items. Further, 11 items were found to have sex-linked response biases, and the overall scale was found to be multidimensional (suggesting that the scale is measuring multiple constructs).Conclusions
The Rasch model appears unsuitable for psychometric evaluation of the SCAT5 symptom checklist. Methods for addressing these issues will be discussed in the context of leveraging this analysis to create a more reliable and valid tool.Keywords:
Differential item functioning
Efficient use of resources often entails the use of the same test across different groups. For example, it is quite common to use one test to measure the language proficiency of students across different age groups or on groups of students with different academic backgrounds. A potential problem of this situation could be test results that are misleading, as some items inadvertently favor one group over another. Differential item functioning (DIF) is the term used when such problems exists in a test at the item (question/prompt) level. This chapter provides readers with an introduction to a Rasch measurement approach to detecting DIF. It also provides readers with an overview of DIF detection methods, a summary of DIF studies in language testing, and an example of a five-stage method to conduct a Rasch measurement approach to detect DIF in a diagnostic English listening test using Winsteps software.
Differential item functioning
Polytomous Rasch model
Cite
Citations (6)
Objective. To further document the measurement properties of each domain of the OA of knee and hip quality of life (OAKHQOL) questionnaire by a Rasch analysis.
Differential item functioning
Polytomous Rasch model
Local independence
Cite
Citations (35)
The assessment of differential item functioning (DIF) remains an area of active research in psychometrics and educational measurement. In recent years, methodological innovations involving mixture Rasch models have provided researchers with an additional set of tools for more deeply understanding the root causes of DIF, while at the same time increased interest in the role of disabilities and accommodations has also made itself felt in the measurement community. The current study furthered work in both areas by using the newly described multilevel mixture Rasch model to investigate the presence of DIF associated with disability and accommodation status at both examinee and school levels for a 3rd grade language assessment. Results of the study found that indeed DIF was present at both levels of analysis, and that it was associated with the presence of disabilities and the receipt of accommodations. Implications of these results for both practitioners and researchers are discussed.
Differential item functioning
Receipt
Cite
Citations (3)
The DyNaChron (Dysfonctionnement Nasal Chronique) questionnaire is a self-reporting 78-item instrument assessing six symptoms and their consequences of chronic nasal dysfunction. Patients complete items of a symptom domain only when it is present but in case the patient presents several or all symptoms, its length can limit its use. Here, we aimed to optimize, or shorten, the DyNaChron for clinical use.A total of 640 patients in 14 rhinology outpatient clinics all over France completed the original DyNaChron questionnaire before the first rhinologic clinic and 15 days later. The optimization process involved Rasch analysis and then qualitative content analyses. Rasch analysis flagged items with a floor/ceiling effect or with important differential item functioning and an expert committee decided whether to retain the flagged items on the basis of clinical importance and statistical characteristics. The psychometric properties of the optimized version were studied according to classical test theory and Rasch analysis.Rasch analysis revealed 4 items with underfit, 6 with an extreme score, 2 that were highly locally dependent and 16 with differential item functioning which 5 of these 16 items were retained after content analysis. In total, 19 flagged items were removed. Factorial analysis confirmed the preservation of the initial instrument structure in the optimized scale; psychometrics properties and scale calibration were the same as or better than the original version.The shortened DyNaChron optimizes the quality of assessment by deleting redundant items and reduces the burden on respondents; the structure is preserved and the psychometrics properties are improved.
Differential item functioning
Polytomous Rasch model
Item analysis
Cite
Citations (2)
This study aimed to assess Rasch-based psychometric properties of the Trypophobia Questionnaire measuring proneness to trypophobia, which refers to disgust and unpleasantness induced by the observation of clusters of objects (e.g., lotus seed pods). Rasch analysis was performed on data from 582 healthy Japanese adults. The results suggested that Trypophobia Questionnaire has a unidimensional structure with ordered response categories and sufficient person and item reliabilities, and that it does not have differential item functioning across sexes and age groups, whereas the targeting of the scale leaves room for improvements. When items that did not fit the Rasch model were removed, the shortened version showed slightly improved psychometric properties. However, results were not conclusive in determining whether the full or shortened version is better for practical use. Further assessment and validation are needed.
Differential item functioning
Polytomous Rasch model
Disgust
Cite
Citations (7)
Background. The Rasch model prescribes procedures to ensure that a scale constructed from multiple items conforms to fundamental requirements of interval scales of measurement. Objective. To test the Rasch properties of the domains of the World Health Organization Quality of Life Instrument–abbreviated version (WHOQOL-BREF) in depressed patients from primary care settings. Design. Cross-sectional, cross-national study. Setting. Primary care. Patients. The sample consisted of 1193 patients having a confirmed diagnosis of depression from 6 countries (Australia, Brazil, Israel, Russia, Spain, and the United States) involved in the Longitudinal Investigation of Depression Outcomes (LIDO) study. Measurements. The Composite International Diagnostic Interview (CIDI) diagnosed depression, whereas the Center for Epidemiological Studies Depression Scale (CES-D) assessed severity of depression. The WHOQOL-BREF measured generic quality of life (QOL). Results. Three of the 4 WHOQOL-BREF domains (physical, psychological, and environment) conform to the Rasch model expectations, following adjustments required mainly because of individual item misfit or differential item functioning (DIF) due to age or country. The social domain showed relatively poor psychometric properties, as observed in previous studies. Conclusions. The Rasch analysis demonstrated that, with some modifications, all domains of the WHOQOL-BREF, except for the social domain, provide an interval scale measure of generic subjective QOL in the context of depressed primary care patients in 6 countries worldwide.
Differential item functioning
CIDI
Depression
Cite
Citations (43)
Differential item functioning
Item analysis
Trait
Cite
Citations (43)
Differential item functioning
Polytomous Rasch model
Cite
Citations (39)
In educational testing, differential item functioning (DIF) statistics must be accurately estimated to ensure the appropriate items are flagged for inspection or removal. This study showed how using the Rasch model to estimate DIF may introduce considerable bias in the results when there are large group differences in ability (impact) and the data follow a three-parameter logistic model. With large group ability differences, difficult non-DIF items appeared to favor the focal group and easy non-DIF items appeared to favor the reference group. Correspondingly, the effect sizes for DIF items were biased. These effects were mitigated when data were coded as missing for item–examinee encounters in which the person measure was considerably lower than the item location. Explanation of these results is provided by illustrating how the item response function becomes differentially distorted by guessing depending on the groups’ ability distributions. In terms of practical implications, results suggest that measurement practitioners should not trust the DIF estimates from the Rasch model when there is a large difference in ability and examinees are potentially able to answer items correctly by guessing, unless data from examinees poorly matched to the item difficulty are coded as missing.
Differential item functioning
Polytomous Rasch model
Local independence
Cite
Citations (8)
Abstract Background Reliable and valid measurement is the foundation of evidence-based practice. The self-administered Foot Health Assessment Instrument (S-FHAI) was recently developed to measure patients’ evaluations of their own foot health. Evidence regarding the psychometric properties of the S-FHAI is limited. The aim of this study was to investigate those properties by using a Rasch analysis. Methods This methodological study analysed secondary data that was collected from nurses ( n = 411) in 2015. The psychometric properties of the S-FHAI were evaluated using the Rasch model. Unidimensionality was analysed first, followed by item functioning, person misfit and differential item functioning (DIF). Results The S-FHAI demonstrated evidence of unidimensionality, with an acceptable item fit according to the Rasch model. Person fit and person separation were low, however, indicating restricted separation among different respondents. Item separation was high, demonstrating clear discrimination between the items. No DIF was detected in relation to gender, but significant DIF was demonstrated in relation to age for 6 of the 25 items. Conclusions The S-FHAI has potential for use in investigating self-reported foot health. The Rasch analysis revealed that the psychometric properties of the instrument were acceptable, although some issues should be addressed to improve the scale. In future, it may be beneficial to analyse the sensitivity of the items and to test the S-FHAI in more diverse patient populations.
Differential item functioning
Polytomous Rasch model
Separation (statistics)
Cite
Citations (11)