Designing a virtual patient dialogue system based on terminology-rich resources: Challenges and evaluation
2019
Virtual patient software allows health professionals to practice their skills by interacting
with tools simulating clinical scenarios. A natural language dialogue system can provide
natural interaction for medical history taking. However, the large number of concepts and
terms in the medical domain makes the creation of such a system a demanding task.
We designed a dialogue system that stands out from current research by its ability to
handle a wide variety of medical specialties and clinical cases. To address the task, we de-
signed a patient record model, a knowledge model for the task, and a termino-ontological
model that hosts structured thesauri with linguistic, terminological and ontological knowl-
edge. We used a frame- and rule-based approach and terminology-rich resources to handle
the medical dialogue. This work focuses on the termino-ontological model, the challenges
involved and how the system manages resources for the French language.
We adopted a comprehensive approach to collect terms and ontological knowledge, and
dictionaries of affixes, synonyms and derivational variants. Resources include domain lists
containing over 161,000 terms, and dictionaries with over 959,000 word/concept entries.
We assessed our approach by having 71 participants (39 medical doctors and 32 non-
medical evaluators) interact with the system and use 35 cases from 18 specialities. We con-
ducted a quantitative evaluation of all components by analysing interaction logs (11,834
turns). Natural language understanding achieved an F-measure of 95.8 per cent. Dialogue
management provided on average 74.3 (±9.5) per cent of correct answers. We performed
a qualitative evaluation by collecting 171 five-point Likert scale questionnaires. All eval-
uated aspects obtained mean scores above the Likert mid-scale point. We analysed the
vocabulary coverage with regard to unseen cases: the system covered 97.8 per cent of their
terms.
Evaluations showed that the system achieved high vocabulary coverage on unseen cases
and was assessed as relevant for the task.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
59
References
11
Citations
NaN
KQI