A deep convolutional neural network to simultaneously localize and recognize waste types in images.

Accurate waste classification is key to successful waste management. However, most current studies have focused exclusively on single-label waste classification from images, which goes against common sense. In this paper, we move beyond single-label waste classification and propose a benchmark for evaluating the multi-label waste classification and localization tasks to advance waste management via deep learning-based methods. We propose a multi-task learning architecture (MTLA) based on a convolutional neural network, which can be used to simultaneously identify and locate wastes in images. The MTLA comprises a backbone network with proposed attention modules, a novel multi-level feature pyramid network, and a group of joint learning multi-task subnets. To achieve joint optimization of waste identification and location, we designed the loss functions according to the concepts of focusing and joint. The proposed MTLA achieved performance similar to that of experts and had high scores for multiple tasks related to waste management. Its F1 score exceeded 95.50% (95.12% to 95.88%, with a 95% confidence interval) on the multi-label waste classification task, and the average precision score was over 81.50% (@IoU = 0.5) on the waste localization task. To improve interpretation, heatmaps were used to visualize the salient features extracted by the MTLA. The proposed MTLA is a promising auxiliary tool that can improve the automation of waste management systems.