Automated Corn Ear Height Prediction Using Video-Based Deep Learning

In corn breeding, hand-measurement of ear height is a labor-intensive process, thus limiting scalability. Here we show that it is feasible to automate estimation of the average ear height of a row of corn in experimental fields used for corn breeding. For this purpose we use point pattern analysis on predicted shank-node locations extracted from video captured on uncalibrated cameras moving through a plot at a fixed height from the ground (4 feet and 2 feet). First, a convolutional neural network-based object detection system (YOLOv3) was trained to detect the ear-stalk connection point and applied to the collected videos. Detected ear position and time information from each frame were super-imposed into a point pattern and point-features were then extracted. Using ridge regression to predict the average ear height per plot, we achieved 0.772 concordance, 2.989 inches root mean squared error, and 2.263 inches mean absolute error compared with hand-measured average ear height. Feature weight importance suggests that one camera may be sufficient for prediction without significant decrease in accuracy. This deep learning system can be utilized by mounting cameras onto the plot combine harvester to collect the necessary videos during harvest and could be expanded to quantify other phenotype measurements of interest that are labor-intensive to collect.