Using UAV-Imagery for Leaf Segmentation in Diseased Plants via Mask-Based Data Augmentation and Extension of Leaf-based Phenotyping Parameters

Abstract

In crop production plant diseases cause significant yield losses. Therefore, the detection and scoring of disease occurrence is of high importance. The quantification of plant diseases requires the identification of leaves as individual scoring units. Diseased leaves are very dynamic and complex biological object which constantly change in form and color after interaction with plant pathogens. To address the task of identifying and segmenting individual leaves in agricultural fields, this work uses unmanned aerial vehicle (UAV), multispectral imagery of sugar beet fields and deep instance segmentation networks (Mask R-CNN). Based on standard and copy-paste image augmentation techniques, we tested and compare five strategies for achieving robustness of the network while keeping the number of labeled images within reasonable bounds. Additionally, we quantified the influence of environmental conditions on the network performance. Metrics of performance show that multispectral UAV images recorded under sunny conditions lead to a drop of up to 7% of average precision (AP) in comparison with images under cloudy, diffuse illumination conditions. The lowest performance in leaf detection was found on images with severe disease damage and sunny weather conditions. Subsequently, we used Mask R-CNN models in an image-processing pipeline for the calculation of leaf-based parameters such as leaf area, leaf slope, disease incidence, disease severity, number of clusters, and mean cluster area. To describe epidemiological development, we applied this pipeline in time-series in an experimental trial with five varieties and two fungicide strategies. Disease severity of the model with the highest AP results shows the highest correlation with the same parameter assessed by experts. Time-series development of disease severity and disease incidence demonstrates the advantages of multispectral UAV-imagery for contrasting varieties for resistance, and the limits for disease control measurements. With this work we highlight key components to consider for automatic leaf segmentation of diseased plants using UAV imagery, such as illumination and disease condition. Moreover, we offer a tool for delivering leaf-based parameters relevant to optimize crop production thought automated disease quantification imaging tools.