Characterizing Crowdsourced Data Collected Using DESIM (Descriptive to Executable Simulation Modeling)

Across many case studies, the Descriptive to Executable Simulation Modeling (DESIM) method has demonstrated the ability to capture and model qualitative knowledge from multiple subject-matter experts (SMEs), convert those models to an executable form using a crowdsourcing approach, and interactively visualize the outputs. This method helps decision makers leverage collective expertise to perform complex “What if?” analysis. This paper takes advantage of a large-scale multiple-model application of DESIM to illustrate the nature and interpretation of the data produced throughout its multiple phases. Lessons learned from this study provide direction toward future evaluation and improvements to this method.