Towards Knowledge-Enhanced Viewing Using Encyclopedias and Model-Based Segmentation

To m ake accu rate deci sions based on i maging data, radiolog ists must as sociate t he viewed i maging data with t he corresponding anatomical structures. Furthermore, given a disease hypothesis possible image findings which verify the hypothesis must be considered and where and how they are expressed in the viewed images. If rare anatomical variants, rare pathologies, unfamiliar protocols, or ambiguous findings are present, external knowledge sources such as medical encyclopedias are con sulted. T hese s ources are acces sed u sing keywords t ypically de scribing a natomical structures, image findings, pathologies. In th is paper we pres ent o ur vision of how a patie nt's i maging data can be a utomatically e nhanced with an atomical knowledge as well as knowledge about image findings. On one hand, we propose the automatic annotation of the images with labels from a standard anatomical ontology. These labels are used as keywords for a medical encyclopedia such as STATdx to acces s an atomical des criptions, information abou pathologies an image findings. On the other h and we envision encyclopedias to contain links to region- and finding-specific image processing algorithms. Then a f inding is evaluated on an image by applying the respective algorithm in the associated anatomical region. Towards realization of our vision, we present our method and results of automatic annotation of anatomical structures in 3D MRI brain images. Thereby we develop a co mplex surface mesh model incorporating major structures of the brain and a model-based segmentation method. We dem onstrate the validity by analyzing the results of several training and segmentation experiments with clinical data focusing particularly on the visual pathway.