Introduction

One of the largest threats in apple orchards is scab. Current procedures involve models based on weather data that predict the likelihood of scab attacks. In case of alarm the orchard is sprayed with preventive pesticides and this typically happens 25-30 times per season. The scab attacks the leaves and stays on fallen leaves that reinfect the trees with rainwater, making it an advantage to include a-priori knowledge on previous infections in the model. When scab attacks are visible by the naked eye after 9-12 days, it is too late for curative measures. There are machine vision systems that detect scab spots on apples after harvest, and few trials on early detection on leaves in laboratory conditions with NIR spectral bands in CCD and InGaAs sensor ranges. The goal for this experiment was to investigate how to inspect leaves while on the trees with CCD cameras, which could make it tangible to do automatic mapping of infected areas in the orchard and possibly early detection in the future.

Methods

Nine untreated shoots were imaged before and after the model warned an almost certain infection. They were imaged 4 days after, which is the threshold for curative measures, and 12 days after where infection was visible to us. Three NIR/Red indices were compared for maximum contrast of scab spots; NDVI, MNDVI and RVI. Two Basler ACE 640-90GM cameras were used with 650nm bandpass filter and polarizing filter on one and 850nm on the other. The polarizing filter was used to improve the correlation between the two colorbands as the surface of a leaf is 5 times more specular reflective in Red than NIR. Also the dynamic range in outdoor daylight was too large so three exposures were used to build HDR images. The 3D processing was done in three different ways (in order of descending magnitude of 3D reconstruction and CPU intensity): 1. Dynamic programming 3D reconstruction using the 2^nd derivative of the images. 2. Assuming the leaves existed in a plane that optimized the correspondence of edge features. 3. Manually panning through planes one by one and detecting leaf features in focus. After finding the depth of leaves the images from each camera was overlaid to produce index images.

The image processing that segmented the image into leaves that were in focus to be inspected for spots was done in the Halcon image processing library by MvTec. The inspiration came from how human perception sees leaves. Eight different texture features were extracted and fed into a neural network classifier. The training took 5 seconds but classification only took 20ms.

Within regions classified as ”leaves in focus” a simple spot detection was performed in Halcon using a large kernel smoothing filter, image subtraction and dynamic threshold.

A monocular approach with 590nm longpass filter on a color camera without NIR cut filter was also tested for the index images. The principle in this setup was that the blue channel represented the NIR and the red channel represented the red after a linear transform known as spectral sharpening.

Results

The best index image was MNDVI because the spotless parts of the leaves were very uniform and the spots were dark. Full 3D reconstruction was noisy causing blurred leaf edges and spots in index image, making the plane based approach more reliable for texture classification and spot detection. With relatively few leaves in the image the automatic depth detection worked well, but in online orchard situations the panning method will be more reliable. Two out of nine leaves had infected spots in the images and were detected by the algorithm on the four day threshold for curative treatment while it was still invisible to the naked eye. After 12 days it was certain that all nine shoots were in fact infected. The texture based segmentation succeeded in separating background such as sky, other rows, and grass from the leaves in focus and at the assumed depth plane and this made it possible to detect spots on the leaves. There were false positives in the edges of the leaves in a few cases. Future research is recommended on detection of these leaf edges so that these false positives can be rejected.

Conclusion