Towards a meta-method for the engineering of situational evaluation methods for domain-specific modeling tools

Peltoniemi, Ari Towards a Meta-Method for the Engineering of Situational Evaluation Methods for Domain-Specific Modeling Tools Jyvaskyla: University of Jyvaskyla, 2014, 116 p. Information Systems Science, Master’s Thesis Supervisor: Leppanen, Mauri Domain-Specific Modeling (DSM) is an approach to Information Systems Development (ISD) in which the abstraction level of development is raised from the solution domain to the problem domain. DSM enables the automation of ISD, particularly in narrow and well-established domains, in which the domain concepts, rules and semantics can be meaningfully specified as constructs of DSM languages. DSM tools provide facilities for DSM language specification and application as well as model transformation. DSM tools are typically evaluated by the industry for the justification of tool acquisitions. DSM tools are also evaluated for research purposes. In order to assure the validity of the results, an evaluation method must address the situational context of the evaluation as well as its multi-disciplinary dimensions. The current literature provides very limited support for the engineering of situational evaluation methods for DSM tools. The primary objective of the study is to investigate how to methodically support the engineering of situational evaluation methods for DSM tools. A practical need for the method support was identified in a case study, in which DSM tools were evaluated in an industrial context. The premise of the study suggests that the application of Situational Method Engineering (SME) principles to the evaluation of DSM tools would provide a potential solution. The Design Science Research (DSR) approach was applied as the research framework for the study. Two artifacts were designed and evaluated, according to the principles of DSR: 1) an evaluation criteria checklist for DSM tools, and 2) a baseline method for the engineering of situational evaluation methods for DSM tools. The checklist is designed for evaluators, to be used as a practical guideline in the situational formulation of the evaluation criteria for DSM tools. The application of the checklist also promotes the commensuration of the evaluation results. The conceptual baseline method is designed to be instantiated by method engineers in the engineering of situational evaluation methods for DSM tools. The main contribution of the study is a design theory or a Meta-Method for the engineering of situational evaluation methods for DSM tools. Meta-Method is conceptually and empirically evaluated. Future research is required to confirm the findings and further elaborate Meta-Method.