Fast Training of POI Recommendation Models Using Gradient Compression

Point-of-Interest (POI) recommendation plays an important role in location-based services. Regarding POI recommendation as a trajectory prediction problem, attention-based models have shown promising achievements to tackle this problem. With the rapid development of LBSN applications, modern training datasets and POI recommendation models are becoming increasingly large and complicated, which results in time-consuming training procedures. Existing solutions adopt data parallel training for POI models, but they still suffer from the high latency of inter-device data communication. To address this issue, we propose a fast training method for POI recommendation models. It conducts a layer-wise top-k gradient compression to reduce the overhead of data transfer. Meanwhile, it modifies the Adam optimizer with an error feedback mechanism to avoid the decrease of POI recommendation accuracy resulting from gradient compression. Experimental results show that our methods can achieve similar even better accuracy than uncompressed methods while the communication time is shorter.