Development of Planted Seedling Area Recognition Algorithm for Vision-based Autonomous Driving Rice Transplanter Development
Embedded Files
1. Motivation
As the agricultural population is gradually decreasing, it is required to develop autonomous driving rice transplanters to cope with the aging problem.
The existing GPS based self-driving rice transplanters sometimes invades the area of seedlings that has already been planted.
Therefore, it is necessary to develop an algorithm that recognizes the area of seedlings from images and controls the heading direction of the rice transplanter.
2. Goal
Develop a deep learning algorithm that recognizes the area of planted seedlings by attaching an RGB camera to a rice transplanter.
Find the boundary between the planted area and the non-planted area.
With the detected boundary line, the autonomous driving rice transplanter is controlled so that it does not invade the planted area.g
(a)
(b)
(c)
Figure 1. (a) Showing a planted area, (b) an unplanted area, (c) a boundary line between the two areas
3. Method
3-1. Dataset
When planting with a rice transplanter, we acquired the planting images using an RGB camera. Figure 2 below is the dataset we used to train the deep neural network.
Figure 2. Dataset used for algorithm development
3-2. Data Augmentation
The amount of acquired images is not enough to train the network because the experiment environment is limited.
Therefore, we used image augmentation to increase the amount of training data.
A python augmentation library called Albumentation is used for our research.
Figure 3 below is an example of image augmentation.
Figure 3. Image augmentation example
3-3. Seedling Area Recognition Network Using U-Net
We decided to use U-Net, which is convenient to use and well known for high performance in the image segmentation field.
After receiving the image and passing through the U-Net, the network parameters were learned through binary cross entropy loss.
Figure 4 shows the flow chart for learning the seedling area recognition network.
Figure 4. Flowchart for training seedling area recognition network and structure of U-Net
The existing U-Net uses batch normalization (BN) for all convolution layers for efficient batch training.
However, we used domain normalization (DN) to reduce the domain gap that occurs because the environment is different for each rice paddy field.
Batch Normalization : A method of normalizing data by Equation (1) using the mean
and standard deviation of all pixel values of N data in one batch
(1)
Figure 5. Area to which Batch Normalization is applied
Domain Normalization : A method of normalizing data by Equation (2) using the mean
and standard deviation of all pixel values in one image
(2)
Figure 6. Area to which Domain Normalization is applied
3-4. Result
Figure 7. Loss curve when using BN
Figure 8. Loss curve when using DN
Figure 9. Area prediction results using BN
Figure 10. Area prediction results using DN
When we used DN, we were able to predict the area of the seedling better than BN.
Through these results, we were able to prove the assumption that the neural network can recognize the area of the seedling.
4. Implementation
As shown in Figure 11, after directly acquiring data, we proceeded with deep neural network training.
Figure 11. Image obtained by attaching a camera to a rice transplanter
After obtaining the result of the trained network as shown in Figure 12, we found the boundary line between the planted area and the non-planted area through image processing.
Figure 12. Result of recognizing the area of seedlings
You can check the loss curve that occurred when training the deep learning algorithm in Figure 13 below.
Figure 13. Train loss and validation loss curve generated when training a deep learning algorithm
※ Terms for validating the performance of the model
TP : If the actual label is True, but the prediction is True
FN : If the actual label is True, but the prediction is False
TN : If the actual label is False, but the prediction is False
FP : If the actual label is False, but the prediction is True
In order to verify the performance of the algorithm, we obtained accuracy, recall, and precision.
Proportion of values that the algorithm correctly predicted labels
among real data
Proportion of values predicted by the algorithm to be True
among values whose actual label is True
Proportion of values whose actual label is True
among the values predicted by the algorithm to be True
Accuracy = 0.974
Recall = 0.946
Precision = 0.985
5. Future work
The task of finding a boundary line between the planted seedling area and the non-planted area was completed.
Therefore, we plan to configure the controller so that the rice transplanter does not invade the planted seedling area while autonomously driving along the boundary line.
# Photo at work
Page updated
Google Sites
Report abuse