Urea Injection Control Based on Deep-Q Networks for SCR Aftertreatment Systems

Abstract The regulations on NOx emissions from diesel vehicles have been stringent in recent years. Various techniques such as lean NOx trap (LNT) and selective catalytic reduction (SCR) have been developed to lessen the NOx emissions. The urea-based SCR method, which utilizes NH3 as reducing agent to remove NOx, is widely used. Determining optimal amount of injected urea that keeps NOx at outlet below regulated NOx emission and also minimizes the amount of dosed urea is important. Model predictive control (MPC) is popularly used to determine the optimal amount of injected urea. However, applying MPC to real vehicle driving may be difficult because the on-line computation of MPC is too costly to be conducted in the engine control unit (ECU), the computation performance of which is significantly low at present. Therefore, reinforcement learning (RL) is considered as an alternative to on-line control method. In this paper, deep Q-networks (DQN), which is an off-policy RL with discrete action space and suitable to solve high dimensional problem, is applied to determine the amount of urea injection in the SCR system. The simulation of urea injection control with DQN has been conducted with respect to inlet NOx emissions of real driving data.