MMSE based joint Tomlinson-Harashima source and linear relay precoder design in non-regenerative MIMO relay system

Abstract This paper proposes a joint nonlinear transceiver design scheme based on minimum mean square error (MMSE) criterion for non-regenerative multiple input multiple output (MIMO) relay system. The proposed scheme decomposes the error covariance matrix, reformulates the original joint design problem as two separate optimization problems, and then provides a closed-form solution with only local channel state information (CSI) available at the source and destination. Performance evaluation shows that the proposed scheme significantly outperforms linear schemes, and has a competitive performance compared with existing global CSI based nonlinear schemes, both iterative and non-iterative.