Remote-Center-of-Motion Recommendation toward Brain Needle Intervention Using Deep Reinforcement Learning

Brain needle intervention is a specific diagnosis and therapy procedure in brain disorders, such as brain tumors and Parkinson’s disease. Preoperative needle path planning is a vital step to guarantee the patient’s safety and reduce lesions. For positioning accuracy in the CT/MRI environment, we have developed a novel needle intervention robot in our previous work. Because the robot is currently designed for the rigid needle, the task of preoperative path-planning is to search for an optimal Remote Center of Motion (RCM) for needle insertion. Therefore, this work proposes an RCM recommendation system using deep reinforcement learning. Considering the robot kinematics, this system takes the following criteria/constraints into consideration: clinical obstacle (blood vessels, tissues) avoidance (COA), mechanically inverse kinematics (MIK) and mechanically less motion (MLM) for the robot. We design a reward function to combine the above three criteria based on their corresponding importance level and utilize proximal policy optimization (PPO) as the main agent of reinforcement learning (RL). RL methods are proved to be competent in searching the RCM, which satisfies the above criteria simultaneously. On the one hand, the results present that RL agents obtain the success rate of finishing the designed task at 93%, which has reached the human level in the tests. On the other hand, the RL agents have the remarkable capability of combining more complex criteria/constraints in future work.