Joint Crop and Tassel Segmentation
Hao Lu, Zhiguo Cao, Yang Xiao, Yanan Li, Yanjun Zhu
Introduction
We describe a segmentation method that can segments two kinds of objects with different visual properties as are the crop and maize tassel. The method has good flexibility and expandability. What we mean by flexibility lies in the ability of the method to tackle different colours simultaneously and expandability is used in the sense that one could incorporate more semantic labels into the model. Overall, we have made three contributions:
- A region-based colour modelling method that integrates region proposals generation and ensemble neural networks is developed for joint crop and maize tassel segmentation.
- A joint segmentation dataset regarding crop and maize tassel is constructed and published online, in the hope that it can serve as a benchmark to facilitate related studies.
- Two general and effective strategies are proposed to accelerate the computation of colour statistics and prediction of ensemble model.
Results show that our method significantly outperforms other state-of-the-art approaches on tassel segmentation, with average precision of 74.3%, and achieves comparable performance of 77.8% on the traditional crop segmentation even with the naivest colour feature (RGB).
Technical Pipeline
Specifically, two recent superpixel segmentation methods are employed to generate region proposals and an ensemble neural networks model is proposed to predict colour labels. Further details please refer to the related publications.
Dataset
The dataset contains three different versions, which are compatible with its corresponding software versions. In particular, in version 2.0, we further include the original images used in the training phase. In version 3.0, we extends significantly more test images with more detailed subsets distinction. Version 3.0 is the main contribution of our journal paper. One is highly recommended to download the version 3.0.
To download the joint segmentation dataset, you agree that:
- the dataset will be used for academic purposes only
- you will cite this paper if you use the dataset in your publication
- you will not distribute the dataset to others
- you must ask permission of the corresponding author if the dataset will be used for commercial purposes
The dataset has three versions, please see the Code section for more details.
Joint-Segmentation-Dataset-v1.0
Joint-Segmentation-Dataset-v2.0
Joint-Segmentation-Dataset-v3.0
Code
Here, one can download a complete implementation of our method. By referring version 1.0, one can exactly reproduce the results reported in our conference paper. Whereas in version 2.0, we has made several modifications. In particular, we replace the original NNI color model implementation that relies on Matlab Neural Network Toolbox with a more flexible self-implementation. Now the learning process of NNI color model becomes completely transparent. Note that the performance of re-implementation is sightly different from our original results reported in the paper due to the different mechanism of regularization. In version 1.0 we depend on early stopping, but in version 2.0, we resort to weight regularization. In addition, the training code for the AP-HI method is also provided. Version 3.0 corresponds to the journal version, which is almost identical to version 2.0. The difference, however, is that we have included our proposed speed-up strategies to accelerate the algorithm.
Note that in version 2.0 and 3.0 software packages, to pursue superiority in terms of running time, we have used many advanced programming skills specific to Matlab, such as vectorization and inline function. If one is interested in improving his Matlab programming skills, we highly suggest you do some in-depth analyses on our code. From the beginning, you may find it hard to write, read, and/or debug vectorized code, because we are all taught by C/C++-style lessons. But once getting familiar with these techniques, you will find there exists similar patterns within these tricks. By writing concise and efficient codes, you will get quick feedbacks from your program, and in return, making progress faster on your project.
Source code and model can be downloaded from:
jointSegment-Matlab-release-v1.0
jointSegment-Matlab-release-v2.0
jointSegment-Matlab-release-v3.0
Publication
Hao Lu, Zhiguo Cao, Yang Xiao, Yanan Li, Yanjun Zhu, Joint crop and tassel segmentation in the wild, in Chinese Automation Congress (CAC), pages 474-479, 2015.
Hao Lu, Zhiguo Cao, Yang Xiao, Yanan Li, Yanjun Zhu, Region-based colour modelling for joint crop and maize tassel segmentation, Biosystems Engineering, volume 147, pages 139-150, 2016.
Compared to its conference version, we have mainly made following extensions in the journal version:
- We extended the dataset with 208 crop and 70 maize tassel images that contain different subsets indicating practical scenarios that may appear in the field-based conditions. Its previous version only contains 50 crop and 30 maize tassel images.
- We enriched our methodology by developing two general and effective strategies to speed up the feature extraction and model prediction process.
- Experiments have been re-conducted based on a more reasonable performance measure. Also, we have included more detailed evaluation and discussion towards algorithms both in effectiveness and efficiency.
Citation
Conference Version
@inproceedings{lu2015joint, title={Joint crop and tassel segmentation in the wild}, author={Lu, Hao and Cao, Zhiguo and Xiao, Yang and Li, Yanan and Zhu, Yanjun}, booktitle={Chinese Automation Congress (CAC)}, pages={474--479}, year={2015}, doi={10.1109/CAC.2015.7382547},}Journal Version
@Article{lu2016jointseg, title={Region-based colour modelling for joint crop and maize tassel segmentation}, author={Lu, Hao and Cao, Zhiguo and Xiao, Yang and Li, Yanan and Zhu, Yanjun}, journal={Biosystems Engineering}, year={2016}, pages={139--150}, volume={147}, doi={10.1016/j.biosystemseng.2016.04.007},}