An Investigation on Reliability of Media Literacy Rubric Within The Scope of Generalizability Theory

The aim of the study is to develop a valid and reliable measurement tool through generalizability theory by considering the errors seen in different variation sources. This scale will be used to test students’ performance on media texts during media literacy course. As inter scorer reliability obtained via generalizability theory was examined, this study can be defined as a basic research. Pre-service teachers studying at the department of Turkish Language Teaching of Mersin University during 2014-2015 spring semester participated in the study. Media literacy rubric was prepared in assessing the identical performance tasks. The data obtained from eight scorers related to six performance tasks in the rubric were analyzed according to generalizability theory and the findings were examined accordingly. After eight scorers evaluated 110 students into seven tasks of the rubric, “o x p x g” pattern was divided into seven variance sources. In addition, generalizability (G) coefficient was found 0.80, and Phi coefficient, 0.73. Considering these results, media literacy rubric proves to be convenient for students’ performance on examining media components. In order to get a standardized rubric, it is important to use the rubric developed in this study for different media tools .