Automated flower counting from partial detections: Multiple hypothesis tracking with a connected-flower plant model

Abstract This paper presents an automated flower counting method based on Multiple Hypothesis Tracking (MHT) with a connected-flower plant model which is based on detections of flowers. Multiple viewpoints of each plant are taken into account as plants are considered in which flowers can occlude each other. To prevent double counting and to solve inconsistencies caused by false flower detections, a model is developed which describes the plant movement with respect to the camera. The uncertainty of the flower detections is considered in this model. To address variations in the velocity of the plant movement, the model realized in this work explicitly takes into account that motions of flowers are correlated since the flowers are connected to each other via the stem of the plant. This is in contrast to the traditional MHT approach where the movement of each object is typically modeled and estimated separately. In our approach, based on the set of detected flowers, the uncertainty of the plant movement is reduced. As a result, the movement of modeled but not always observed flowers is still properly tracked. To demonstrate the validity of the approach, the proposed counting method is tested on a dataset obtained in a real greenhouse containing multiple viewpoints of 71 Phalaenopsis plants and compared to existing methods. The methods considered include a single viewpoint approach, a heuristic state of the practice approach and an MHT approach with both an independent and connected object description. Within a margin of 1 flower, these methods respectively counted the number of flowers in 44 % , 58 % , 70 % and 92 % of the plants correctly. As a result, this work validates the superiority of the MHT approach with a connected-flower plant model.