Reliability, Validity, and Reader Acceptance of LI-RADS— An In-depth Analysis

Rationale and Objectives This study aimed to analyze interreader agreement and diagnostic accuracy of Liver Imaging Reporting and Data System (LI-RADS) in comparison to a nonstandardized 5-point scale and to assess reader acceptance of LI-RADS for clinical routine. Materials and Methods Eighty-four consecutive patients at risk for hepatocellular carcinoma who underwent liver magnetic resonance imaging were included in this Health Insurance Portability and Accountability Act-compliant retrospective study. Four readers rated the likelihood of hepatocellular carcinoma for 104 liver observations using LI-RADS criteria and a 5-point Likert scale (LIKERT) based on subjective impression in two separate reading sessions. Interreader agreement was assessed using kappa statistics (κ). Diagnostic accuracy was assessed with receiver operating characteristic analysis. Reader acceptance was evaluated with a questionnaire. A sub-analysis of LI-RADS's major features (arterial phase hyper-enhancement, washout, capsule appearance, and threshold growth) and scores for lesions 1.5 cm was performed. Results LI-RADS showed similar overall interreader agreement compared to LIKERT (κ, 0.44 [95%CI: 0.37, 0.52] and 0.35 [95%CI: 0.27, 0.43]) with a tendency toward higher interreader agreement for LI-RADS. Interreader agreement (κ) was 0.51 (95%CI: 0.38, 0.65) for arterial phase hyper-enhancement, 0.52 (95%CI: 0.39, 0.65) for washout, 0.37 (95%CI: 0.23, 0.52) for capsule appearance, and 0.50 (95%CI: 0.38, 0.61) for threshold growth. Overall interreader agreement for LI-RADS categories was similar between observations 1.5 cm. Overall diagnostic accuracy for LIKERT and LI-RADS was comparable (area under the receiver operating characteristic curve, 0.86 and 0.87). Readers fully agreed with the statement “A short version of LI-RADS would facilitate the use in clinical routine” (median, 5.0; interquartile range, 2.25). Conclusions LI-RADS showed similar interreader agreement and diagnostic accuracy compared to nonstandardized reporting. However, further reduction of complexity and refinement of imaging features may be needed.