Framework for Evaluating Knowledge-Based Systems

As discussed in the previous chapter, a number of different knowledge-based system (KBS) development models have been proposed. These vary depending on such things as the size and complexity of the knowledge-based system and how well requirements for the system are understood. Evaluation models are necessarily tied to system development models. Because the same development model is not appropriate for use with all knowledge-based systems (Hilal & Soltan, 1993), evaluation models will also vary from one knowledge-based system to another. This means that one generic, all-purpose evaluation model is not appropriate for all knowledge-based systems. Instead of a general evaluation model, we present a framework to help the evaluator construct his or her own set of evaluations that is tailored to the development model being used. The framework will help the evaluator decide what evaluation issues to address, when in the development life cycle the issues should be addressed, what evaluation methods to use for what issues at which stage, and who should supply the evaluation data.