Argue, observe, assess: Measuring disciplinary identities and differences through socio-epistemic discourse

Calls for interdisciplinary collaboration have become increasingly common in the face of large-scale complex problems (including climate change, economic inequality, and education, among others); however, outcomes of such collaborations have been mixed, due, among other things, to the so-called “translation problem” in interdisciplinary research. This article presents a potential solution: an empirical approach to quantitatively measure both the degree and nature of differences among disciplinary tongues through the social and epistemic terms used (a research area we refer to as discourse epistemetrics), in a case study comparing dissertations in philosophy, psychology, and physics. Using a support-vector model of machine learning to classify disciplines based on relative frequencies of social and epistemic terms, we were able to markedly improve accuracy over a random selection baseline (distinguishing between disciplines with as high as 90% accuracy) as well as acquire sets of most indicative terms for each discipline by their relative presence or absence. These lists were then considered in light of findings of sociological and epistemological studies of disciplines and found to validate the approach's measure of social and epistemic disciplinary identities and contrasts. Based on the findings of our study, we conclude by considering the beneficiaries of research in this area, including bibliometricians, students, and science policy makers, among others, as well as laying out a research program that expands the number of disciplines, considers shifts in socio-epistemic identities over time and applies these methods to nonacademic epistemological communities (e.g., political groups).