Embedded Files
An effective perception system is a fundamental component for farming robots, as it enables them to properly perceive the surrounding environment and to carry out targeted operations. The most recent approaches make use of state-of-the-art machine learning techniques to learn an effective model for the target task. However, those methods need a large amount of labelled data for training. A recent approach to deal with this issue is data augmentation through Generative Adversarial Networks (GANs), where entire synthetic scenes are added to the training data, thus enlarging and diversifying their informative content.
An effective perception system is a fundamental component for farming robots, as it enables them to properly perceive the surrounding environment and to carry out targeted operations. The most recent approaches make use of state-of-the-art machine learning techniques to learn an effective model for the target task. However, those methods need a large amount of labelled data for training. A recent approach to deal with this issue is data augmentation through Generative Adversarial Networks (GANs), where entire synthetic scenes are added to the training data, thus enlarging and diversifying their informative content.
In this work, we propose an alternative solution with respect to the common data augmentation techniques, applying it to the fundamental problem of crop/weed segmentation in precision farming. Starting from real images, we create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with synthesized counterparts. To do that, we employ a conditional GAN (cGAN), where the generative model is trained by conditioning the shape of the generated object. Moreover, in addition to RGB data, we take into account also near-infrared information, generating four channel multi-spectral synthetic images.
In this work, we propose an alternative solution with respect to the common data augmentation techniques, applying it to the fundamental problem of crop/weed segmentation in precision farming. Starting from real images, we create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with synthesized counterparts. To do that, we employ a conditional GAN (cGAN), where the generative model is trained by conditioning the shape of the generated object. Moreover, in addition to RGB data, we take into account also near-infrared information, generating four channel multi-spectral synthetic images.
Examples of SPADE RGB output
Examples of SPADE NIR output
Paper
Paper
M. Fawakherji, C. Potena, A. Pretto, D.D. Bloisi, and D. Nardi
Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming
In: Robotics and Autonomous Systems, Volume 146, December 2021
Paper: https://www.sciencedirect.com/science/article/pii/S0921889021001469
Pre-print on Arxiv: https://arxiv.org/abs/2009.05750
BibTeX:
@article{fppbnRAS2021,
title = {Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming},
author = {Mulham Fawakherji and Ciro Potena and Alberto Pretto and Domenico D. Bloisi and Daniele Nardi},
journal = {Robotics and Autonomous Systems},
volume = {146},
pages = {103861},
year = {2021},
issn = {0921-8890},
doi = {https://doi.org/10.1016/j.robot.2021.103861}}
Code
Code
Dataset
Dataset
We performed experiments on three publicly available datasets, considering two different types of crops, namely sugar beet and sunflower.
Sugar beet: For sugar beet, we used two publicly available datasets: the Bonn dataset and the Stuttgart, both datasets have been collected by using a BOSCH Bonirob farm robot moving on a sugar beet field across different weeks. The datasets are composed of images taken by a 4-channels (RGB + NIR) JAI AD-13 camera, mounted on the robot and facing downward.
Sunflower: In this work, we introduce a new dataset for crop/weed segmentation, called Sunflower dataset that has been collected by the authors of this paper. Data has been acquired by using a custom-built agricultural robot moving in a sunflower farm in Jesi, Italy. The dataset has been recorded in spring, across a period of one month, starting from the emergence stage of the crop plants, until the end of the useful period for the use of chemical treatments. As for the Bonn and the Stuttgart datasets, images were acquired using a 4-channels (RGB + NIR) JAI AD-13 camera, mounted on the robot and facing downward. The Sunflower dataset, composed of 500 images, provides RGB and NIR images with pixel-wise annotation of 3 classes: crop, weed, and soil.
Page updated
Report abuse