Detection of Gray Mold Infection in Plants Using a Multispectral Imaging System

Gray mold disease caused by the fungus Botrytis cinerea damages many crop hosts worldwide and is responsible for heavy economic losses. Early diagnosis and detection of the disease would allow for more effective crop management practices to prevent outbreaks in field or greenhouse settings. Furthermore, having a simple, non-invasive way to quantify the extent of gray mold disease is important for plant pathologists interested in quantifying infection rates. In this paper, we design and build a multispectral imaging system for discriminating between leaf regions, infected with gray mold, and those that remain unharmed on a lettuce (Lactuca spp.) host. First, we describe a method to select two optimal (high contrast) spectral bands from continuous hyperspectral imagery (450-800 nm). We then built a system based on these two spectral bands, located at 540 and 670 nm. The resultant system uses two cameras, with a narrow band-pass spectral filter mounted on each, to measure the multispectral reflectance of a lettuce leaf. The two resulting images are combined using a normalized difference calculation that produces a single image with high contrast between the leaves’ infected and healthy regions. A classifier was then created based on the thresholding of single pixel values. We demonstrate that this simple classification produces a true positive rate of 95.25% with a false positive rate of 9.316%.


27
Botrytis cinerea is an airborne plant pathogen that causes losses by means of gray mold disease in over 28 200 different species of crops worldwide, with the most damage being seen in dicotyledonous hosts [1]. 29 Typical symptoms of the disease observed in plant leaves and fruits include soft rots followed by the 30 appearance of gray masses of conidia [1]. A color image of various Salinas and US96 lettuce (Lactuca 31 sativa) leaves with severely advanced infection is shown in Figure 1. 32 Gray mold is also to be a major threat to greenhouse-grown plants, where control of the disease is 33 usually challenging and expensive [2]. Careful application of fungicides would allow for effective 34 management of gray mold when growing plants. Using fewer well-timed sprays of fungicide is more 35 efficient than more frequent applications through harvest due to the possibility of promoting resistant 36 populations of B. cinerea [3]. An imaging system that can highlight areas of gray mold infection in plant 37 leaves with accuracy and consistency would allow for quick and easy assessment of the disease on a large 38 number of plants at once so that growers can make informed decisions on spraying fungicide. In addition, 39 as demonstrated here, such an imaging system provides a useful tool for plant pathologists studying the 40 spread of gray mold. The ability to consistently and quantitatively detect the first signs of B. cinerea and 41 to monitor disease progression in a leaf can provide detailed trait information to researchers comparing 42 treatment methods or investigating genotype-driven differences in susceptibility. 43 Several studies have addressed gray mold diagnosis and detection in plants. Traditionally In this paper, we demonstrate a process for selecting optimal spectral bands to be integrated in a two-72 band multispectral camera imaging system and evaluate its performance, for the purposes of detecting B. 73 cinerea infection on L. sativa leaves to monitor disease progression. In section 2, we outline the theory 74 needed to assess the hyperspectral data, while in section 3, we describe the experimental setup used to 75 acquire our hyperspectral and multispectral data. Section 4 describes the results of our timelapse 76 experiments, while section 5 ends with a conclusion. 77

79
Hyperspectral and multispectral imaging devices have been widely used for measuring irradiance spectra 80 of a scene with varying spectral resolution, range, and acquisition methods. They measure light collected 81 from a scene as a function of two spatial dimensions and one spectral dimension. The main distinction 82 between a hyperspectral and multispectral system is the quantity of spectral bands measured by the 83 device. There are a variety of methods for acquiring this three-dimensional data that are mostly described 84 by the way they discriminate the spatial and spectral dimensions [10]. Examples of spatial data 85 acquisition methods include whiskbroom (point-scanning) and pushbroom (line-scanning) systems, as 86 well as "snapshot" two-dimensional imagers [10]. Some ways these devices acquire spectral data include 87 spectral filtering, dispersive elements like prisms or diffraction gratings, and interferometric Fourier 88 transform methods [10]. 89 To discover the optimal spectral bands, we use a prism-based pushbroom scanning hyperspectral 90 imaging camera that has been calibrated per Ref.
[11] to provide comprehensive lettuce irradiance 91 spectra. Relative reflectance was calculated to quantify both healthy and infected lettuce leaf tissues, such 92 that 93 where I sample is the intensity measured from the lettuce, I tile is the downwelling intensity measured from a 95 white (99% reflectivity) spectralon tile illuminated by the source, x n and y m are the x and y coordinates 96 corresponding to the discrete integer spatial position n and m, respectively, and λ k is the k th wavelength 97 element in the datacube. For visualization and labeling purposes, 2-dimensional (2D) panchromatic 98 imagery was generated by integrating our 3-dimensional (3D) data cube along the spectral axis for each 99 image slice, such that 100 After reflectivity has been calculated, we leveraged graphical reflectance indexing to establish a 102 simple analysis of the scene's spectral measurements. By taking radiance measurements at specific 103 spectral components, correlations between features of the scene can be made. A common example of this 104 is the normalized difference vegetation index (NDVI) which takes the difference of reflectance of two 105 spectral bands in the near infrared (NIR) and red regions and divides by their sum [12]. Generally, the 106 contrast, ν , from such two-band calculations for arbitrary wavelengths, where the intensity measurements I are implicitly dependent upon x n and y m for clarity. This procedure 109 minimizes the effect of lighting and atmospheric conditions on measurements to enable more accurate 110 comparison [13]. Also, image data from two spectral bands can be condensed into one single image that 111 is produced from the contributions of both bands. 112 The goal for a system implementing reflectance indices is to select two spectral bands where the 113 contrast is optimized between two characteristics of interest, a and b (e.g., symptomatic versus healthy 114 areas of lettuce leaves). Using data from the hyperspectral reflectance plots, a heat map, which is defined 115 by 116 was obtained by sweeping through each combination of two spectral bands, in H, the optimal spectral bands can be determined for use in a multispectral camera system. 120

121
To calculate v over many combinations of spectra, a pushbroom hyperspectral imaging camera was used 122 to collect a three-dimensional spatial and spectral data cube over a visible to near-infrared spectral range 123 with wavelength resolution spanning 0.5 to 2.4 nm for 500 to 800 nm light. Lettuce leaves of the variety 124 Black Seed Simpson were inoculated with B. cinerea which spread throughout the leaves over the course 125 of a few days. The leaves were placed on a damp paper towel, inside a 10 in x 10 in bioassay dish 126 (Corning). Treatment leaves were inoculated with B. cinerea suspended in Potato dextrose broth (PDB). 127 Control leaves, in the same tray, were mock-inoculated with PDB alone. When the disease symptoms 128 were significant (about 50 percent coverage of the leaf area) on most of the inoculated lettuce leaves, the 129 tray was imaged by the pushbroom camera, and a tungsten light source was used to illuminate the sample 130 leaves. The pushbroom camera and tungsten light source setup is presented in Figure 2. A tungsten 131 halogen lamp was configured to illuminate the target at a normal angle of incidence from the leaf's 132 surface normal. Our pushbroom spectral camera was positioned 40 cm away, yielding a spatial resolution 133 of approximately 6.5 mm. Light entered the objective lens, slit, and collimator where it was dispersed 134 using a prism. Dispersed light was then imaged onto a focal plane array (FPA) using a reimaging lens and 135 raw data were calibrated in accordance to Ref. [11]. The FPA measures an image with a spatial 136 dimension along the slit and a spectral dimension caused by the dispersive prism. In order to acquire the 137 other spatial dimension required for a two-dimensional image, the entire imaging system is mounted on a 138 rotating platform that moves while the camera acquires successive frames that form the three-dimensional 139 data cube, I sample . 140 After acquiring the hyperspectral datacubes, two-dimensional images of the lettuce were obtained 141 using Eq. (2.2). Infected and control (healthy) tissues, from inoculated and control leaves, respectively, 142 were manually labeled by defining regions of interest (ROIs). Each ROI was then spatially averaged to 143 create a normalized reference spectrum, which are depicted in Figure 3. Generally, the normalized 144 reflectivity is the same except around 540 nm and 720 nm. Our reflectance measurements presented in 145 The reference spectra were further used to produce the heat map in Figure 4. It should be noted that 148 the heat map is symmetric about the diagonal due to the same spectral bands used in the calculation from 149 Eq. (2.4). From these results, the highest contrast is located at λ 1 = 670 nm and λ 2 = 530 nm. 150 Using this information, we selected two spectral band-pass filters for use in a dual-camera 151 multispectral system. The filters selected had center pass-bands at 540 nm and 670 nm filter and a 10 nm 152 full width at half maximum spectral bandwidth. Note that one filter (540 nm) deviates slightly from our 153 calculated maximum due to off-the-shelf filter availability at the time of purchase; however, this is only 154 expected to reduce v by 2.8%. 155 A schematic of the 2-band imaging system is depicted in Figure 5 (a). Light from the sample first 156 enters a beam splitter (BS), which directs light from a scene to each camera. Transmitted or reflected light 157 from the BS then enters the spectral band-pass filters BP1 or BP2, respectively, before forming an image 158 by L1 or L2 onto FPA1 or FPA2, respectively. The FPA consisted of two BlackFly 1.3 megapixel, 30 159 frames per second USB3 monochrome cameras. Both cameras were configured to take images of the 160 scene simultaneously. A photograph of the completed experimental setup is depicted in Figure 5 (b). 161 Since this system was used to take lettuce measurements over several days, a white LED source was used 162 that does not radiate as much heat as the tungsten lamp in order to lessen environmental stress on the 163 lettuce leaves. In addition, experiments were performed inside of a clear plexiglass container prevent the 164 spread of B. cinerea to the surrounding environment. Since the imaging system was located outside of 165 the plexiglass barrier, glare off its surface was reduced by placing the LED off-axis in an illumination 166 geometry similar to darkfield microscopy [15]. 167 In post-processing, an affine image registration algorithm [16] aligns the two images, taking into 168 account translational and rotational differences. In addition, both of these images are normalized to flat 169 field images taken prior to measurements with the same camera settings. This corrects for non-uniform 170 spectral radiance from the LED and differences in shutter time and aperture size in the cameras. Finally, 171 the calculation from Eq. (2.3) is applied using the spatially registered images as input arguments. To 172 enable rapid segmentation of the individual leaf's boundaries while maintaining moisture, they were 173 placed on a red terrycloth background. This provided a high positive value of v that could be easily 174 distinguished from the lower values produced by the diseased or healthy tissues. The background pixel 175 values are set to global minimum values during post-processing so that the leaves in the contrast images 176 stand out better visually. 177

178
The multispectral camera system was used to obtain contrast images of two distinct species -Lactuca 179 sativa cv. Salinas and L. serriola acc. US96UC23 (US96), parents of a recombinant inbred mapping 180 population [17]. The leaves of each species were inoculated with B. cinerea and images acquired over 181 several days as the symptoms spread. Figure 6 shows the contrast images of a US96 and Salinas leaf over 182 seven days alongside a reference color image taken using a color USB camera. It can be seen that the 183 healthy lettuce areas have lower values of v as compared to damaged areas, as expected from our 184 hyperspectral results given healthy tissue yields a lower reflectance at 670 nm compared to 540 nm 185 (producing a negative value), while infected tissue causes the reflectivity at both wavelengths to become 186 more similar (producing a value closer to 0). 187 Additional processing was implemented to measure the percent area of the lettuce leaves that were 188 covered by the gray mold disease. Each leaf was segmented from the background by thresholding pixel 189 values. By using a bright red background in the images, the contrast between the band-pass images was 190 much higher than any measurements on a particular leaf. Using this relation, the leaves can be isolated 191 from the image by classifying pixel values less than a specified background threshold as belonging to 192 leaves. This was used to create a binary mask of the original contrast image that was used to isolate 193 single leaves for further processing. Within each leaf, the diseased and healthy areas were segmented by 194 further thresholding the pixel intensities. In order to determine a reasonable threshold to best discriminate 195 between healthy and symptomatic lettuce areas, ROIs were labeled in a set of contrast images to compare 196 the healthy and symptomatic values. 197 A histogram of v for both the healthy and symptomatic pixel classes are presented in Figure 7. Using 198 the symptomatic and healthy means (μ s and μ h ) and standard deviations (σ s and σ h ), a decision threshold, 199 η , was defined by estimating a probability density function (PDF) for each type and calculating the 200 location where the two PDFs were equal. Based on this calculation, it was determined that a contrast 201 threshold of η = -0.0915 was sufficient for discriminating healthy and symptomatic tissues. 202 In Figure 8, a receiver operating characteristic (ROC) curve was created based on the aforementioned 203 PDFs. From this curve, it can be seen that for a probability of false positive classification, P F , of 204 symptomatic tissue equal to 0.09316, the true positive probability, P D , is 0.9525. 205 A binary classification image that uses this decision threshold is depicted in Figure 9. There are some 206 false-positive areas in the binary imagery that are indicated as belonging to a diseased lettuce region when 207 they are not. These areas occur around the leaves' edges where the red background was classified as part 208 of the leaf due to shadowing effects and was not removed from the image, and along the leaf veins which 209 can tend to have more yellow coloration than the leaf blades. To account for this when calculating the 210 amount of disease symptoms on a particular leaf, an offset subtraction described by 211 where N raw,s (t) is the number of pixels classified as symptomatic at time, t, and t 0 is the time of the first 213 binary image immediately of inoculation. N net,s (t) is the net number of symptomatic pixels in the binary 214 image after t hours. 215 The ratio of the diseased leaf area to the total leaf area was calculated and is presented in Figure  216 10 (a) and (b) for Salinas and US96, respectively. Both the diseased area percentage with and without 217 offset subtraction in Eq. (3.1) are provided in the plots. This provides a data metric for studying the 218 infection's spread and leaf damage over time. According to these plots, the symptoms spread at what 219 appears to be an exponential rate until the leaf is close to fully damaged. At this point, the rate of disease 220 spread starts to decrease, consistent with a population growth curve approaching the carrying capacity. 221

222
A methodology for building a multispectral imaging system with the purpose of locating gray mold 223 infection in plant leaves was demonstrated. A pushbroom hyperspectral imaging system was leveraged to 224 select two narrow spectral band-pass filters to use in the dual-camera system. Post-processing of the 225 collected data provides a basis for visually identifying infection areas and quantifying the spread over 226 time. By taking the normalized difference between the two band-pass images, a significant contrast 227 between leaf areas with gray mold disease and unaffected areas was observed, and the computation 228 required little complexity. A simple threshold was used to programmatically discriminate between 229 infected and healthy leaf areas with reasonable accuracy. 230

231
The authors would like to thank the United States Department of Agriculture (USDA) for funding this 232 study as part of the National Needs Graduate Fellowships Program (2016-38420-25324: 233 "Multidisciplinary Graduate Training in Advanced Technologies for High Yield Sustainable 234 Agriculture"). We also appreciate the assistance from Katherine Denby, University of York, in helping 235 establish B.cinerea infections, and Richard Michelmore, UC Davis, for providing the lettuce genotypes 236 for the study.

Receiver Operator Characteristic for Lettuce Infection Classification
Optimal P F 0.09316 Optimal P D 0.9525