CReBot: Exploring interactive question prompts for critical paper reading

Pre-compiled guidelines with a static question list can stimulate critical thinking while reading a scientific paper. However, they could be less engaging than taking live question prompts from others. In this paper, we develop CReBot that interactively asks section-level critical thinking questions and customize it for routine paper readers with prior research experience and novices new to research. Our first within-subjects study with 24 routine readers demonstrates CReBot’s engagement and usefulness over static guidelines. Then, with more teacher-like question-specific hints prepared for CReBot, we conduct another within-subjects study with 20 novices. The results, however, indicate that CReBot might not be better than static guidelines for beginners. Nevertheless, both user groups favor CReBot’s contextualized questions and interaction flexibility. We conclude with design implications for interactive tools to facilitate critical reading.