Olaf Köller

Leibniz Institute for Science and Mathematics Education

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Jürgen Baumert

Max Planck Institute for Human Development

Ulrich Trautwein

University of Tübingen

Oliver Lüdtke

Leibniz Institute for Science and Mathematics Education

Jens Møller

Kiel University

Gabriel Nagy

Leibniz Institute for Science and Mathematics Education

Kai Maaz

DIPF | Leibniz Institute for Research and Information in Education

Johanna Fleckenstein

University of Hildesheim

Wilfried Bos

Max Planck Institute for Human Development

Jan Retelsdorf

Universität Hamburg

Friederike Zimmermann

Kiel University

Cooperative Institutions

Deutsche Nationalbibliothek

Leibniz Institute for Science and Mathematics Education

FH Münster

DIPF | Leibniz Institute for Research and Information in Education

Kiel University

Max Planck Institute for Human Development

Max Planck Society

Humboldt-Universität zu Berlin

Universität Hamburg

TU Dortmund University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

The merits of representational pictures in educational assessment: Evidence for cognitive and motivational effects in a time-on-task analysis

Contemporary Educational Psychology (2017)

Marlit Annalena Lindner Oliver Lüdtke Simon Grund Olaf Köller

Adding representational pictures (RPs) to text-based items has been shown to improve students' test performance. Focusing on potential explanations for this multimedia effect in testing, we propose two functions of RPs in testing, namely, (1) a cognitive facilitation function and (2) a motivational function. We found empirical support for both functions in this computer-based classroom experiment with N = 410 fifth and sixth graders. All students answered 36 manipulated science items that either contained (text-picture) or did not contain (text-only) an RP that visualized the text information in the item stem. Each student worked on both item types, following a rotated within-subject design. We measured students' (a) solution success, (b) time on task (TOT), and identified (c) rapid-guessing behavior (RGB). We used generalized and linear mixed-effects models to investigate RPs' impact on these outcome parameters and considered students' level of test engagement and item positions as covariates. The results indicate that (1) RPs improved all students' performance across item positions in a comparable manner (multimedia effect in testing). (2) RPs have the potential to accelerate item processing (cognitive facilitation function). (3) The presence of RPs reduced students' RGB rates to a meaningful extent (motivational function). Overall, our data indicate that RPs may promote more reliable test scores, supporting a more valid interpretation of students' achievement levels.

Facilitation

10.1016/j.cedpsych.2017.09.009

Cite

Citations (35)

PISA und die Bildungsverantwortung der Evangelischen Kirche

VS Verlag für Sozialwissenschaften eBooks (2005)

Annette Scheunpflug Claudia Standfest Olaf Köller

10.1007/978-3-322-80658-1_6

Cite

Citations (1)

Pictures in Test Items: Effects on Response Time and Response Correctness

Applied Cognitive Psychology (2011)

Steffani Saß Jörg Wittwer Martin Senkbeil Olaf Köller

Summary Although pictures are often added to text in items of educational tests, little is known about their influence on item solving. Therefore, we conducted an experiment in which we examined how pictures affected item solving. A total of N = 158 fourth‐grade students completed a physics knowledge test under one of six experimental conditions. The experimental conditions varied according to whether or not pictures were presented in the stem and in the answer options of the test items. The results showed that pictures in the stem and in the answer options increased the correctness with which students responded to the test items. This was particularly true for test items that required the application of relationships. In addition, response time was reduced when pictures were added to the answer options of the test items. Hence, pictures are an important feature of test items that produce changes in item processing. Copyright © 2011 John Wiley & Sons, Ltd.

Multiple choice

10.1002/acp.1798

Cite

Citations (44)

Multiple-Choice-Prüfungen an Hochschulen?

Zeitschrift für Pädagogische Psychologie (2015)

Marlit Annalena Lindner Benjamin Strobel Olaf Köller

Zusammenfassung. Multiple-Choice-Aufgaben (MCA) sind bei der Leistungsmessung großer Personengruppen besonders ökonomisch. Im Zuge des hohen Prüfungsaufkommens im Bachelor-Master-System werden MCA-Klausuren auch an deutschen Hochschulen immer häufiger eingesetzt. Doch welche diagnostische Qualität haben Prüfungen mit MCA und wo liegen Vorteile und Probleme? In diesem Literaturüberblick kommen wir zu vier zentralen Ergebnissen: (1) MCA von hoher Qualität sind in vielen Fällen diagnostisch vergleichbar zu Constructed-Response-Aufgaben. (2) Es existieren effektive Strategien, um Rateeffekten zu begegnen. (3) Der Einfluss des Prüfungsformats auf Lern- und Prüfungsstrategien ist kaum vermeidbar. (4) Besonders geeignet für den Hochschulkontext sind die MC-Formate Multiple-Response und Multiple-True-False sowie insbesondere computerbasierte Testaufgaben. Zusätzlich zeigen wir einen Mangel an Forschungsarbeiten auf, die für belastbare Aussagen über den diagnostischen Wert von MCA in realen Kontexten unerlässlich sind und leiten daraus Forschungsfragen ab.

10.1024/1010-0652/a000156

Cite

Citations (30)

Bildungsstandards

Springer eBooks (2016)

Olaf Köller

10.1007/978-3-531-20002-6_26-1

Cite

Citations (1)

Professionswissen von Lehramtsstudierenden der mathematisch-naturwissenschaftlichen Fächer - Testentwicklung im Rahmen des Projekts KiL

Thilo Kleickmann Jörg Großschedl Ute Harms Aiso Heinze Stefanie Herzog

Berichtet wird uber das Projekt Messung professioneller Kompetenzen in mathematischen und naturwissenschaftlichen Lehramtsstudiengangen (KiL), dessen primare Zielsetzung es ist, Instrumente zur Erfassung des universitar erworbenen Professionswissens bei Lehramtsstudierenden mit mathematisch-naturwissenschaftlichen Fachern zu entwickeln. Es wird skizziert, wie die Tests zum bildungswissenschaftlichen Wissen sowie zum Fachwissen und fachdidaktischen Wissen in den Fachern Biologie, Chemie, Mathematik und Physik konstruiert wurden, und es werden erste Ergebnisse aus den Testanalysen mit insgesamt 1240 Studierenden vorgestellt. (ZPID).

Source

Cite

Citations (11)

Transparency: A Necessary Requirement for the Construct Validity of Assessment Centres

Applied Psychology (1996)

Martin Kleinmann Constanze Kuptsch Olaf Köller

La validité de construction des centres d'évaluation et la performance perçue des candidats sont-elles sensibles au fait que ceux-ci aient ou non connaissance des dimensions jugées pertinentes? 119 étudiants de 1° cycle ont participéà une session de centre d'évaluation dans des conditions de transparence et de non transparence. II est apparu que la validité de construction était supérieure quand les participants connaissaient les dimensions recherchées et le comportement attendu de leur part. En outre, les notations des sujets de la condition "transparence" qui admettaient avoir soumis leur conduite aux dimensions attendues bénéficiaient d'une validité discriminante et convergente supérieure à celle des notations des sujets qui affirmaient n'avoir tenu aucun compte des dites dimensions. L'impact de la transparence des dimensions recherchées sur la validité de construction des centres d'évaluation est discuté. The present study tested whether the construct validity of assessment centres and the perceived performance of candidates were influenced by whether or not the requirement dimensions were made transparent for participants. A total of 119 college students participated in an assessment centre under either transparent or nontransparent conditions. Results showed that construct validity was higher when participants knew the requirement dimensions and what behaviour was required of them. In addition, the ratings of subjects in the transparency condition who reported that they had oriented their behaviour towards the requirement dimensions had higher convergent and discriminant validity than the ratings of subjects who reported that they disregarded the requirement dimensions. The consequences of the transparency of requirement dimensions for the construct validity of assessment centres are discussed.

10.1080/026999496378800

Cite

Citations (8)

The relationship between individual characteristics and practice behaviour within an adaptive arithmetic learning program

Journal of Computer Assisted Learning (2023)

Anna Hilz Karin Guill Janina Roloff Karen Aldrup Olaf Köller

Abstract Background Although research on mathematics learning programs has taken off in recent years, little is known about how different person characteristics are related to practice behaviour with such programs. When implementing a mathematics learning program in the classroom, it might be important to know whether students with specific characteristics need more encouragement to make use of this learning opportunity. Objectives Using a supply‐use model, we analysed the predictive power of students' cognitive characteristics (prior mathematics knowledge), personality traits (conscientiousness), motivational‐affective characteristics (mathematics self‐concept and mathematics anxiety), and family background characteristics (socioeconomic status and migration background) for practising with an adaptive arithmetic learning program. Methods We used longitudinal data from 203 fifth graders from seven non‐academic‐track schools in Germany. Practice behaviour, measured by completed tasks in the learning program, was recorded weekly for every student over a period of 22 weeks. Results and Conclusions The results of our multilevel analyses showed that mathematics anxious students practised less with the program. We did not find any relationship with the other characteristics. Takeaways Our results suggest that mathematics anxious students need more encouragement when practising with a mathematics learning program; otherwise, they do not get the chance to benefit from the use.

Conscientiousness

Predictive power

Mathematical anxiety

10.1111/jcal.12780

Cite

Citations (7)

The 2I/E model: Integrating temporal comparisons into the internal/external frame of reference model.

Journal of Educational Psychology (2018)

Fabian Wolff Gabriel Nagy Jan Retelsdorf Friederike Helm Olaf Köller

The 2I/E model is an extension of Marsh’s (1986) classical internal/external frame of reference (I/E) model. Whereas the classical I/E model describes the formation of academic self-concept by the joint operation of social (external) and dimensional (internal) comparisons, the 2I/E model integrates temporal comparisons as an additional internal comparison process. In the 2I/E model, paths from achievement levels to self-concepts in the same subject are assumed to represent social comparisons, paths from achievement levels to self-concepts in another subject are assumed to represent dimensional comparisons, and paths from achievement changes to self-concepts in the same subject are assumed to represent temporal comparisons. The present research validated the 2I/E model in six empirical studies with a total of more than 20,000 students. Generally, there were strong positive social comparison paths, moderate negative dimensional comparison paths, and small positive temporal comparison paths. Furthermore, the relations of the 2I/E model were replicated with both grades and standardized test scores as achievement indicators (Study 1 and Study 2). Dimensional comparison effects were stronger between dissimilar subjects, like math and German, than between similar subjects, like German and English (Study 2 and Study 3). Also, the relations of the 2I/E model were shown when prior self-concepts were controlled for (Study 4). Overall, the 2I/E model seems to have the potential to enrich future theory and research on self-concept formation and comparison processes. (PsycINFO Database Record (c) 2019 APA, all rights reserved)

Frame of reference

10.1037/edu0000319

Cite

Citations (35)

The effect of achievement composition on STEM competencies: Analyzing longitudinal data of German students’ scientific and computer literacy in secondary school.

Journal of Educational Psychology (2024)

Jana Kähler Inga Hahn Olaf Köller

Scientific Literacy

10.1037/edu0000885

Cite

Citations (1)