Weakly Supervised Region of Interest Extraction Based on Uncertainty-Aware Self-Refinement Learning for Remote Sensing Images

Region of interest (ROI) extraction plays a significant role in the field of remote sensing image (RSI) processing. Recently, weakly supervised ROI extraction methods have attracted considerable attention due to low labeling cost. Most of them follow the pipeline of first generating pseudo labels and then using the pseudo labels to train a segmentation model. However, there remain problems to be solved: 1) the unbalanced distribution of foreground and background samples in the RSI dataset influences the network performance; 2) the pseudo labels mainly cover the most discriminative part of object regions that are incomplete; and 3) training with pseudo labels inevitably causes noise issues that degrade the model performance. To solve these issues, we propose a weakly supervised uncertainty-aware self-refinement learning (UASRL) method, where the initial unbalanced image-level labels are progressively refined to high-quality pixel-level annotations. In the proposed UASRL, we first present a deep generative model combined with self-attention modules to improve the unbalanced distribution in the weakly labeled dataset. Then, we design a confidence-weighted complementary erasing-based weakly supervised method to generate pseudo labels with high integrity. Finally, for training with noisy pseudo labels, we develop an uncertainty-aware joint optimization (UAJO) training strategy to reduce the negative effect caused by noisy labels and further refine pixelwise labels in a coarse-to-accurate manner, which in turn jointly promotes the model’s performance. Extensive experiments on three types of RSI datasets reveal that our proposed method is superior to other competing methods and shows a preferable tradeoff between annotation cost and detection performance.