Detection of Three Fruit Maturity Stages in Wild Blueberry Fields Using Deep Learning Artificial Neural Networks

Abstract. Wild blueberry (Vaccinium angustifolium Ait.) fields are almost exclusively picked by mechanical harvesting machines which collect all available berries from the bushes in a single operation during late summer. The resulting stream of harvested fruit may contain a significant fraction of unmarketable unripe green and red berries in addition to the marketable fully ripe blue berries. In this study, we evaluated the performance of machine vision to detect wild blueberry fruit maturity on the bushes before harvesting. Fruit maturity data from unharvested blueberry fields could be used in decision-support tools for optimizing the harvest date as part of an integrated research program for improving wild blueberry harvester efficiency. Four versions of the Yolo (You only look once) family of convolutional neural networks on the Darknet deep learning framework were trained to recognize three classes of fruit maturity (unripe green, unripe red, and ripe blue) on a deep learning server equipped with a graphics processing card. The training dataset of 4,220 color images were cropped from 211 high-resolution digital camera images collected in a commercial wild blueberry field at four dates up to the time of harvest in summer 2018. The images contained overhead views of blueberry bushes with berries, which were labeled according to the three stages of maturity. A second wild blueberry field was used for independent validation of the results at four additional dates. The YoloV3-spp network performed best with 91% recall, and 28.3 s inference time, and could successfully identify the three maturity stages of blueberries in the second field.