Deep Learning Assisted Sum-Product Detection Algorithm for Faster-than-Nyquist Signaling

A deep learning assisted sum-product detection algorithm (DL-SPA) for faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed detection algorithm concatenates a neural network to the variable nodes of the conventional factor graph of the FTN system to help the detector converge to the a postenor probabilities based on the received sequence. More specifically, the neural network performs as a function node in the modified factor graph to deal with the residual intersymbol interference (ISI) that is not modeled by the conventional detector with a limited number of ISI taps. We modify the updating rule in the conventional sum-product algorithm so that the neural network assisted detector can be complemented to a Turbo equalization. Furthermore, a simplified convolutional neural network is employed as the neural network function node to enhance the detector’s performance and the neural network needs a small number of batches to be trained. Simulation results have shown that the proposed DL-SPA achieves a performance gain up to 2.5 dB with the same bit error rate compared to the conventional sum-product detection algorithm under the same ISI responses.