Inter-rater reliability of the Play Assessment for Group Settings

AbstractPlay Assessment for Group Settings (PAGS) was originally designed to measure the play ability of 2- to 8-year-old children in their natural day-care context and for the use of different professionals. The main purpose of this study was to examine the inter-rater reliability of the PAGS. Twelve raters from different professional backgrounds scored a total of 78 children, both videotaped co-scored cases and individual live cases. The many-faceted Rasch model (MFR) was used to examine the goodness-of-fit of the rater severity estimates. Of the 12 raters, 91.7% were demonstrated to fit the MFR model expectations. Overall the results of this study indicated that the raters scored the PAGS in a reliable way, and that they behaved like independent experts. The separation index (2.09) of the raters indicated that the inter-rater scorings were comparable with each other for practical purposes. Further research on the different response processes of children in different diagnostic groups is needed.