Towards a Human-AI Hybrid Framework for Inter-Researcher Similarity Detection

Understanding the intellectual landscape of scientific communities and their collaborations has become an indispensable part of research per se. In this regard, measuring similarities among scientific documents can help researchers to identify groups with similar interests as a basis for strengthening collaboration and university-industry linkages. To this end, we intend to evaluate the performance of hybrid crowd-computing methods in measuring the similarity between document pairs by comparing the results achieved by crowds and artificial intelligence (AI) algorithms. That said, in this paper we designed two types of experiments to illustrate some issues in calculating how similar an automatic solution is to a given ground truth. In the first type of experiments, we created a crowdsourcing campaign consisting of four human intelligence tasks (HITs) in which the participants had to indicate whether or not a set of papers belonged to the same author. The second type involves a set of natural language processing (NLP) processes in which we used the TF-IDF measure and the Bidirectional Encoder Representation from Transformers (BERT) model. The results of the two types of experiments carried out in this study provide preliminary insight into detecting major contributions from human-AI cooperation at similarity calculation in order to achieve better decision support. We believe that in this case decision makers can be better informed about potential collaborators based on content-based insights enhanced by hybrid human-AI mechanisms.