Collaborative Probabilistic Semantic Mapping Using CNN

Performing collaborative semantic mapping is a critical challenge for cooperative robots to enhance their comprehensive contextual understanding of the surroundings. The chapter bridges the gap between the advances in collaborative geometry mapping that relies on pure geometry information fusion, and single robot semantic mapping that focuses on integrating continuous raw sensor data. In this chapter, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formulated in a distributed setting. The key novelty of this work is the modelling of the hierarchical semantic map fusion framework and its mathematical derivation of its probability decomposition. At single robot level, the semantic point cloud is obtained by a fusion model combining information from heterogeneous sensors and is used to generate local semantic maps. At collaborative robots level, local maps are shared among robots for global semantic map fusion. Since the voxel correspondence is unknown in collaborative robots level, an Expectation-Maximization approach is proposed to estimate the hidden data association. Then, Bayesian rule is applied to perform semantic and occupancy probability update. The experimental results on the UAV (Unmanned Aerial Vehicle) and UGV (Unmanned Ground Vehicle) platforms show the high quality global semantic map, demonstrating the accuracy and utility in practical missions.