A method of ROC analysis by applying item response theory (IRT) to results of 1/0 judgments on the presence or absence of abnormal findings in CT image readings

The purpose of this study is to develop a method of ROC analysis to evaluate both the ability of individual readers to detect abnormal findings and the detectability of abnormal findings in individual cases by applying item response theory to the results of 1/0 judgments on presence of abnormal findings in CT image readings. The validity of the method was verified by the following data and methods. Twenty-four readers searched for abnormal findings in 25 cases for which there were chest CT images with defined abnormal findings. From the 1/0 judgment data for the 25 cases with CT images (column) read by the 24 readers (row), each reader's potential ability to detect the abnormal findings (θ), the rate of "1" judgment by each reader, i.e. confidence level for TP and FP, P(θ), and the individual image response characteristic curves with the image as the item were calculated, from which ROC curves that represent the ability of each reader to detect abnormal findings were created. In addition, from the 1/0 judgment data for the 25 cases with CT images (row) read by the 24 readers (column), the potential detectability of abnormal findings for each CT image (θ) and the rate of "1" judgment for the image by readers, i.e. confidence level for TP and FP, P(θ), were calculated, from which ROC curves that represent the detectability of the abnormal finding in each case were created.