PANet: Pixelwise Affinity Network for Weakly Supervised Building Extraction From High-Resolution Remote Sensing Images

To save large human efforts to annotate pixel-level labels, weakly supervised semantic segmentation (WSSS) with only image-level labels has attracted increasing attention. For WSSS, generating high-quality class activation maps (CAMs) is crucial to obtain pseudo labels for training an accurate building extraction model. To generate high-quality CAMs, many existing methods make use of multiscale context fusion of individual entities. Although these methods have shown an improvement on weakly supervised building extraction, they do not take account of the global interrelations beyond individual entities, resulting in inconsistent activated values in CAMs for different building objects. In this study, we develop a pixelwise affinity network (PANet) for weakly supervised building extraction based on image-level labels. We model and enhance the interrelations between building objects by leveraging reliable interpixel affinities, thus optimizing the generation of the CAMs. Moreover, we propose a consistency regularization loss to further refine the generated CAMs on the accuracy of boundary regions. Experiments on two public datasets (InriaAID dataset and WHU dataset) verify the effectiveness of the proposed PANet. Experimental results also show that our method achieves excellent results with over 0.57 points in intersection-over-union (IOU) score and over 0.73 points in F1 score on both datasets and outperforms the comparing methods.