Problematic Tools: Introduction to Symposium on Set Theory in Social Science

Analysts who developed the set-theoretic comparative method (STCM) have formulated admirable goals for researchers who work in the qualitative and multi-method tradition. This method includes above all Charles Ragin’s innovative approach of Qualitative Comparative Analysis (QCA). However, the analytic tools employed by STCM have in many ways become an obstacle to achieving these admirable goals. For example, the system of fuzzy-set scoring appears to be problematic, poorly matched to a standard understanding of conceptual structure, and perhaps unnecessary in its present form. Computer simulations suggest that findings suffer from serious problems of stability and validity; and while the choice of simulations that match the method is a matter of some controversy, the cumulative weight of simulation results raises major concerns about STCM’s algorithms — i.e., its basic, formalized analytic procedures. Questions also arise about the cumbersome formulation of findings in what is often a remarkably large number of causal paths. Relatedly, some scholars question STCM’s rejection of the parsimonious findings, in the form of “net effects,” routinely reported in other methodological traditions. Regarding applications, readily available software has encouraged publication of dozens of articles that appear to abandon key foundations of the method and rely far too heavily on these algorithms. Finally, STCM appears inattentive to the major, recent rethinking of standards and procedures for causal inference from observational data. These problems raise the concern that the set-theoretic comparative method, as applied and practiced, has become disconnected from the underlying analytic goals that motivated Charles Ragin to create it.