Ontology-guided machine learning outperforms zero-shot foundation models for cardiac ultrasound text reports
Sharmila SubramaniamSara RizviRamya RameshVibhor SehgalBrinda GurusamyHasan ArifJeffrey TranRitu ThammanEmeka AnyanwuRonald MastouriG. Burkhard MackensenRima Arnaout
0
Citation
18
Reference
10
Related Paper
Abstract:
Abstract Big data can revolutionize research and quality improvement for cardiac ultrasound. Text reports are a critical part of such analyses. Cardiac ultrasound reports include structured and free text and vary across institutions, hampering attempts to mine text for useful insights. Natural language processing (NLP) can help and includes both statistical- and large language model based techniques. We tested whether we could use NLP to map cardiac ultrasound text to a three-level hierarchical ontology. We used statistical machine learning (EchoMap) and zero-shot inference using GPT. We tested eight datasets from 24 different institutions and compared both methods against clinician-scored ground truth. Despite all adhering to clinical guidelines, institutions differed in their structured reporting. EchoMap performed best with validation accuracy of 98% for the first ontology level, 93% for first and second levels, and 79% for all three. EchoMap retained performance across external test datasets and could extrapolate to examples not included in training. EchoMap’s accuracy was comparable to zero-shot GPT at the first level of the ontology and outperformed GPT at second and third levels. We show that statistical machine learning can map text to structured ontology and may be especially useful for small, specialized text datasets.Keywords:
Ground truth
Our previous study (Ito & Ikegami, in press) hypothesized and confirmed that people would predominantly draw corresponding mental state inference (i.e., inferring mental states correspondent with behaviors in terms of evaluative connotations) from undesirable behavior, whereas they draw not just correspondent but non-correspondent mental state inference as well for desirable behavior. That study, however, used hypothetical behavioral events. Our present study examined whether this asymmetrical inference is evident in the case of inference from real-life behavioral episodes. Participants were asked to remember desirable and undesirable behavior performed by other persons, and to infer the actors’ mental states from their behaviors. The results supported the hypothesis, indicating that, while people’s inclinations to infer correspondent mental states from both behaviors were potent, inference of non-correspondent mental states from desirable behaviors were more frequent than inference from undesirable behaviors. The results also provided insight into the process of idea generation in mental state inference.
Mental model
Cite
Citations (0)
To support the personalized mobile recommendation service, a context modeling approach based on ontology was proposed. The role of context ontologies in mobile recommenders was discussed, and under the application background of petrol station recommendation, a two-level context ontology model was designed. The model contains an upper ontology to describe general concepts, and a domain ontology extended from the upper ontology. The context ontology provides formalizations representing the mainly entities including users, objects and contexts, and their interactive relationships in mobile recommendation environments. Finally, a use case of the instantiated context ontology was demonstrated.
Context model
Context awareness
Ontology Inference Layer
Cite
Citations (0)
Based on the study of the technology of grid and ontology,we imported the new Ontology technology into manufacturing domain,brought out the essential for the importing of Ontology into manufacturing data gird and classified the Ontology into domain Ontology and application ontology.The research will help to the further development of domestic manufacturing industry.
Cite
Citations (0)
Ground truth
Common ground
Cite
Citations (4)
We compare methodologies for trainable document image content extraction, using a variety of ground-truth policies: loose, tight, and pixel-accurate. The goal is to achieve pixel-accurate segmentation of document images. Which ground-truth policy is the best has been debated. ``Loose'' truth is obtained by sweeping rectangles to enclose entire text blocks etc, and can be an efficient manual task. ``Tight'' truth requires more care, and more time, to enclose individual text lines. Pixel-accurate truth, in which only foreground pixels are labeled, can be obtained by applying the PARC PixLabeler tool; in our experience this tool was as quick to use as loose truthing. We have compared the accuracy of all three truthing policies, and report that tight truth supports higher accuracy than loose truth, and pixel-accurate truth yields the highest accuracy. We have also experimented on morphological expansions on pixel-accurate truth, by expanding sets of foreground pixels morphologically, and report that expanded pixel-accurate truth supports higher accuracy than pixel-accurate truth.
Ground truth
Figure–ground
Cite
Citations (7)
보행 분석은 사람 식별, 소아마비, 척수손상, 뇌졸중 환자 등의 운동 평가 및 재활, 퇴행성 뇌질환 진단 등 다양한 분야에서 활용되고 있다. 보행 분석에 있어서 모션 캡쳐의 Ground-Truth 데이터는 광학식 방식을 이용한 Vicon 또는 Optotrak 시스템 등으로부터 산출된 데이터이나, 이들 시스템은 마커를 부착해야하는 점과, 설치 공간상의 제약 및 상당한 고가의 시스템이라는 단점을 갖고 있다. 최근 보행 분석에 있어서 이러한 단점을 보완하고자 Microsoft 사의 Kinect 센서 여러개로 구성된 시스템으로부터 3D 공간에서 body point 위치 데이터를 추출하는 연구가 진행 중이다. 그러나 기존 연구는 제안 시스템의 유효성 평가에 있어서 Ground-Truth 데이터와 제안 시스템 산출 데이터 간의 선형상관관계, 재현가능성, 일치성 등을 증명할 뿐, Kinect 데이터를 기반으로 Ground-Truth 데이터를 모델링하고 그 값을 예측하는 연구는 아직 미비한 상황이다. 본 연구에서는 다변수 다중회귀분석을 이용하여 다중 Kinect 시스템으로부터 산출된 데이터를 기반으로 Ground-Truth 데이터를 모델링하고 예측하는 방법을 고찰하였고, 모델값과 실제 Ground-Truth 데이터값 간의 결정계수와 급내상관계수를 계산함으로써 상관관계에 대하여 분석하였다.
Ground truth
Common ground
Cite
Citations (0)
In the field of multi-view people localization, only a few works consider a non-planar ground surface. In this article we introduce a framework for collecting ground truth data in such case, we show characterization of specific errors and introduce a method to automatically merge multiple ground truth data generated by different users to form a more reliable reference ground truth. We use this reference ground truth to evaluate the error rate, the accuracy and the recall of subjects (6 laymen and 3 with domain knowledge). We show that even laymen can work accurately, but even subjects with domain knowledge miss a number of people in a crowded scene. Our findings show that creating ground truth data requires special attention in this field.
Ground truth
Merge (version control)
Common ground
Cite
Citations (11)
Ontology construction aims to build conceptual knowledge in such a way that the relations among major concepts can be explicitly identified and presented in a machine operable way so as to assist in intelligent processing of computer applications. An upper-level ontology includes general concepts that are used broadly across different domains whereas ontologies acquired by computing through algorithms automatically are more likely to be domain specific. This paper first introduces domain specific core ontology ( mid-level ontology) and application domain ontology ( lower-level ontology) . Then,it presents a top-down approach to build a core ontology for Chinese in the IT domain based on the English upper level ontology SUMO and other English-Chinese resources available. The paper also introduces a bottom-up approach to build domain specific ontology using corpus based approach.
Ontology Inference Layer
Cite
Citations (0)
Web Ontology Language
Ontology language
Context awareness
Cite
Citations (6)
The accuracy of a binarization algorithm is often calculated relative to a ground truth image. Except for synthetically generated images, no ground truth image exists. Evaluating binarization on real images is preferred. The ground truthing between and among different operators is compared. Four direct metrics were used. The variability of the results of five different automatic binarization algorithms were compared to that of manual ground truth results. Significant variability in the ground truth results was found.
Ground truth
Cite
Citations (44)