AB1103 AUTOMATIC FINGER JOINT BONE EROSION SCORE PREDICTION CONSIDERING 2-TIME-POINT X-RAYS OF PATIENTS WITH RHEUMATOID ARTHRITIS BY DEEP LEARNING

Background: Medical image analysis using deep learning (DL) has been attracting attention. In previous research, we proposed a DL method for detection of joint region and evaluation for bone destruction at a single point in time in hand X-rays of patients with rheumatoid arthritis (RA) [1-2]. However, in the score of van der Heijde-modified total Sharp scores (mTSS) in X-rays, it is difficult to apply the method as it is. In mTSS, score difference between 2-time points is important, and there is a problem that the score at each time varies depending on the doctor who evaluates. Objectives: We aimed at developing an mTSS scoring method considering 2-time-point difference with a DL method. Methods: A total of 104 X-ray image sets of both hands at two time points with an interval of ≥1 year were randomly obtained from patients with RA who had visited our clinic in 2015. Well-trained doctors determined the erosion scores of MP and PIP/IP joints of each hand in X-rays according to mTSS. These evaluations of hand joints were performed using our developed annotation software tool. In the learning phase, joint images were randomly divided into five sets for 5-fold cross-validation. We utilized a convolutional neural network model, such as SSD [3], for detecting joint regions and classifying the scores (Fig 1). The models for classification were designed in consideration of the difference in erosion scores of each patient between the 2-time points of X-rays. The loss function of the DL model was defined bellow; SCE: softmax cross entropy MSE: mean squared error t: training data y: output of DL model 0: the former time point 1: the latter time point T: transpose of matrix Here, the coefficient γ is designed to reduce the error for another set of scores with equal differences. The first term of the loss function works to optimize the score at each time point, and the second term works to optimize the score difference at both time points. Thus, our method can be trained without being affected by characteristic training data. Results: The number of joints with differences in erosion score between the former and latter time points was 1 (-2 points), 9 (-1), 2015 (0), 32 (+1), 17 (+2), and 6 (+3). There were no joints with score changes of -5, -4, -3, +4, and +5 points. As a performance of predicting the difference in erosion score between the 2-time points of each patient’s X-ray, our models presented a mean error of 0.412 per each joint in one set for 5-cross validation as compared with physicians’ evaluation (Fig 2). Conclusion: Our DL-based models to predict hand joint erosion scores in X-rays were developed with relatively small samples. This suggests that the predictive performance may increase by collecting more training dataset. Next, we will apply our method to the prediction of joint space narrowing score. References: [1]Izumi K, Hashimoto M, Suzuki K, et al. Detecting Hand Joint Ankylosis in Radiographic Images Using Deep Learning: A Step in Developing Automatic Radiographic Scoring System for Bone Destruction. Arthritis Rheumatol 2018;70 (suppl 10). [2]Izumi K, Suzuki K, Hashimoto M, et al. SAT0543 AUTOMATIC DETECTION OF HAND JOINT REGION, ANKYLOSIS AND SUBLUXATION IN RADIOGRAPHIC IMAGES USING DEEP LEARNING: DEVELOPMENT OF ARTIFICIAL INTELLIGENCE-BASED RADIOGRAPHIC EVALUATION SYSTEM FOR BONE DESTRUCTION. Annals of the Rheumatic Diseases 2019;78 (suppl 2), pp. 1364-1364. [3]Liu W, Anguelov D, Szgedy C, et al. SSD: single shot multibox detector. European Conference on Computer Vision (ECCV) 2016. Acknowledgments: Izumi and Suzuki are contributed equally. Disclosure of Interests: Keisuke Izumi Grant/research support from: Asahi Kasei Pharma, Takeda Pharmaceutical Co., Ltd., Speakers bureau: Asahi Kasei Pharma Corp, Astellas Pharma Inc., Bristol Myers Squibb, Chugai Pharmaceutical Co., Ltd., Eli Lilly Japan K.K., Mitsubishi Tanabe Pharma Co., Kanata Suzuki Employee of: Fujitsu Laboratories Ltd., Masahiro Hashimoto: None declared, Toshio Endoh Employee of: Fujitsu Laboratories Ltd., Kentaro Doi Employee of: Fujitsu Ltd., Yuki Iwai Employee of: Fujitsu Ltd., Yuko Kaneko Speakers bureau: AbbVie, Eisai Pharmaceutical, Chugai Pharmaceutical Co., Ltd., Bristol Myers Squibb, Astellas Pharma Inc., Mitsubishi Tanabe Pharma Co., Pfizer Japan Inc., Janssen Pharmaceutical K.K., Eli Lilly Japan K.K., Santen Pharmaceutical Co., Ltd., Kyowa Hakko Kirin Co. Ltd. and UCB Japan Co. Ltd., Masahiro Jinzaki: None declared, Shigeru Ko Grant/research support from: Fujitsu Ltd., Tsutomu Takeuchi Grant/research support from: Astellas Pharma Inc, Chugai Pharmaceutical Co, Ltd., Daiichi Sankyo Co., Ltd., Takeda Pharmaceutical Co., Ltd., AbbVie GK, Asahikasei Pharma Corp., Mitsubishi Tanabe Pharma Co., Pfizer Japan Inc., Eisai Co., Ltd., AYUMI Pharmaceutical Corporation, Nipponkayaku Co. Ltd., Novartis Pharma K.K., Teijin, Consultant of: Astra Zeneca K.K., Eli Lilly Japan K.K., Novartis Pharma K.K., Mitsubishi Tanabe Pharma Co., Abbivie GK, Nipponkayaku Co.Ltd, Janssen Pharmaceutical K.K., Astellas Pharma Inc., Taiho Pharmaceutical Co. Ltd., Chugai Pharmaceutical Co. Ltd., Taisho Toyama Pharmaceutical Co. Ltd., GlaxoSmithKline K.K., UCB Japan Co. Ltd., Speakers bureau: Astellas Pharma Inc., Bristol Myers Squibb, Chugai Pharmaceutical Co., Ltd., Mitsubishi Tanabe Pharma Co., Pfizer Japan Inc., Santen Pharmaceutical Co., Ltd., Takeda Pharmaceutical Co., Ltd., Teijin Pharma Ltd., AbbVie GK, Asahi Kasei Pharma Corp., Taisho Toyama Pharmaceutical Co., Ltd., SymBio Pharmaceuticals Ltd., Janssen Pharmaceutical K.K., Celltrion Inc., Nipponkayaku Co. Ltd., and UCB Japan