Optimal treatment of chronic kidney disease with uncertainty in obtaining a transplantable kidney: an MDP based approach

Chronic kidney disease (CKD) is one of the most serious and prevalent health issues all over the world. The evolution of CKD can last for many years until the death of patients, and the method of treatment mainly includes medication, dialysis, and transplantation with the evolution of the disease. It has been validated by many empirical studies that for severe CKD patients, the optimal treatment is transplantation if a suitable kidney is available, otherwise the patients should initiate dialysis at a suitable time. It has also been validated that the initiation time of dialysis significantly impacts not only the direct treatment results, but also the success of a future possible kidney transplantation. Motivated by this consideration, we investigate the decision-making problem of the optimal treatment approach to maximize the patient’s total reward including pre-transplant reward and post-transplant reward (if applicable), considering the possibility of having a suitable kidney transplantation in the future. A Markov decision process model is established in which the status of the process is described by the patient health status. We present some structural properties of the decision-making problem, which are used to choose the optimal treatment approach in different health status of patients. We collect the clinical data in the simulation experiments to obtain the fitted curves of the evolution process of different CKD patients, and compare the simulation results with the actual clinical data to demonstrate the advantage of our model.