Introduction
Optimal harvesting date and predicted yield are valuable information when farming open field tomatoes, making harvest planning and work at the processing plant much easier. Monitoring growth during tomato?s early stages is also interesting to assess plant stress or abnormal development. Yet, it is very challenging due to the colours and the high degree of occlusion by the vegetation. We chose to capture and process pictures of the tomatoes during the whole development while keeping costs as low as possible. This work is part of a larger collaborative project, MCUBE, aiming at conceiving a cost effective way of capturing and processing large sets of data in the field.
Optical device
The estimation of the tomato size requires the use of two cameras in the field, each capturing one useful image every day. In order to avoid a complete 3D reconstruction and make the segmentation task more robust, we assume that a tomato can be approximated by a sphere in the 3D space, this sphere projecting into an ellipse in the image plane. The processing algorithm detects and segments the tomatoes, then makes an estimation of their size.
Segmentation algorithm
- updating the tomato position, through linear filtering and optimization procedure,
- computing candidate ellipses from gradient features, using RANSAC estimation. A priori knowledge regarding the size and the shape of the tomato is incorporated.
- adding region (intensity) information in order to select the best ellipse and detect the occlusion areas,
- applying a new active contour model with elliptic shape constraint, in order to refine the segmentation. The shape constraint is introduced as a regularization term in the energy functional of the model, in order to be robust with respect to shadows and occlusions.
- providing four elliptic approximations to the operator, using the obtained contour and gradient information.
Experimental results: The same setup was used for three agricultural seasons (2011-2013). We have identified 21 tomatoes, covering different sites and different years, thus ensuring variability (614 images). In order to evaluate our algorithm, we compared manual segmentations to automatic ones, by calculating a mean square error expressed as a percentage of the average of the ellipse axes. Results were studied regarding the amount of occlusion. For the images with low occlusions (<30%), very good results were obtained with an average mean square error less than 10% for 87% of the images and a low standard deviation, demonstrating both the accuracy and the robustness of the proposed method. Good results were also obtained on 73% of the images with more occlusions (between 30% and 50%).
Radius estimation
Methods. The system calibration is performed at the beginning of the season, using a simple chess pattern placed at several positions in the field. The calibration procedure is then fully automatic and provides the camera parameters. Knowing the elliptic approximations of the tomato in the left and right images, a triangulation procedure recovers the center of the sphere in the 3D space. Then geometrical properties and an optimization procedure enable us to estimate the sphere radius from the ellipse points.
Results. In the experiments conducted in laboratory, the radius estimation error was less than 5% for 91% of the cases. The volume is then estimated from the radius, by applying a correction coefficient (determined experimentally) as tomatoes are not perfect spheres. The error percentage between the estimated and the actual volume is less than 15% for 87% of the cases.
Conclusion
This paper presents an innovative algorithm used for monitoring tomatoes in open fields, consisting of two main parts: segmentation and partial metric reconstruction. Both steps were validated separately using images acquired during three complete agricultural seasons. The complete automatic system has been validated on the images acquired in 2013 (10 tomatoes). We obtained a correct estimation of the tomato radius in 80% of the cases (error less than 10% corresponding to about 0.2 cm). Segmentation imprecision that results from the quality of the images acquired in real conditions is the main source of error. Future work will focus on the implementation of the algorithms on an innovative gateway/platform system (Machine to Machine (M2M) architecture), the goal being to get an operational and ergonomic system that enables farmers to monitor easily the tomato growth.
