Ridge-Following Control
for Agricultural Tractors

In this study, an algorithm for autonomous driving of small agricultural tractors on various types of ridges was developed and the performance was verified through experiments. The algorithm was constructed using a deep learning network and depth images to generalize the performance. And the verification process of the algorithm has been completed in various ridge environments.

Motivation

In recent years, in Korea's agricultural society, the decrease in the labor force due to the aging of the agricultural population has become serious, and thus agricultural productivity is decreasing. Therefore, the demand for autonomous agricultural machines to continuously maintain agricultural productivity is increasing.

Among the various farming methods that can be performed with a tractor, the furrow irrigation method is a farming method in which several crops are grown on a ridge formed in a field. The farming method includes a sowing operation in which humans drive along the ridge and uses a rear-mounted implement to plant seedlings or seeds. In this work, the human controls the steering of the tractor for a very long time. If the autonomous sowing operation using the autonomous tractor is implemented, it will be possible to reduce the labor required for the agricultural operation. In this study, the ridge-following control algorithm of the tractor was developed using the vision-based ridge centerline recognition algorithm.

There are several previous studies on vision-based agricultural land environment perception. However, most of the research is about recognition algorithms using only RGB images. These algorithms can have poor performance in visual environmental changes. In this study, we propose a ridge-following driving algorithm based on stereo depth images and deep learning networks. This method can generalize the performance of the algorithm to various environmental changes according to the amount of sunlight and the type of soil.

Goals

In this study, we develop the ridge centerline estimation algorithm and the tractor control algorithm.

• Development of ridge centerline estimation algorithm using deep learning network and stereo vision .

• Development of tractor steering angle control algorithm that follows the estimated center line of the ridge.

• For depth images acquired from various types of ridges, the error of the ridge centerline estimation algorithm is within 5% of the total horizontal pixels. In addition, the performance difference of the algorithm for each ridge type should not be large.

• The driving path to which the ridge tracking driving algorithm is applied must be within 10cm of the reference path. Also, the performance difference of the algorithms for each ridge type should not be large.

Deep Learning based Ridge Centerline Estimation Algorithm

1) Deep learning network model configuration

The deep learning network structure for estimating the ridge centerline was constructed based on MobileNetV2. MobileNetV2 is a network model that has been studied focusing on reducing the weight of the network. It uses a depthwise separable convolution layer with less computational complexity compared to a general convolution layer. This network was selected because the computing power of the processor to be included in the tractor is lower than that of a typical desktop computer.

A depth image with a size of 398x224 is used as an input to the network, and information about the centerline path is output. For the loss function in the learning process, the Root Mean Square Error between the network output coordinates and the ground truth coordinates was used.

2) Dataset configuration

Depth image collection and annotation were performed to compose a dataset required for deep learning model training and validation. Depth images were collected through a stereo camera attached to a tractor in various ridge environments, and the correct value was marked on the center line of the ridge. In addition, Image Data Augmentation, which increases the diversity of the data set, was applied to generalize the performance of the model to be trained. Through this process, a total of about 20,000 pieces of learning data, verification data, and experimental data were constructed.

3) Training results

Table 1 above shows the RMSEs for the validation dataset and the test dataset, which are expressed as 0.0100 and 0.0105, respectively. Since the output from the constructed deep learning model is a normalized value on a scale of 0 to 1, the average error for each dataset is 1.00% and 1.05% of the number of horizontal pixels of the image. This is a very small value considering that a person directly checked the image and marked the correct value, and it was confirmed that it was within 5% of the quantitative target of performance.

Steering control algorithm of tractor to follow the center line of the ridge

In order to follow the center line of the ridge of the tractor, a PD controller based on the Preview-distance concept was used. A stereo camera that acquires an image of the front of the tractor is installed in the center of the tractor. Therefore, the y-axis direction was set as the current heading angle of the tractor based on the x-coordinate value corresponding to the center of the horizontal pixel of the image.

Experimental results and discussion

The proposed algorithm was applied to the tractor system, and algorithm verification experiments were conducted in various types of ridge environments. In a field environment, straight ridges, curved ridges, and vinyl ridges were molded and used for algorithm verification. In addition, a system for driving the algorithm was constructed. As shown in Figure 13, this system includes a Single Board Computer, a Stereo Camera to obtain information about the ridge in front of the tractor, and a CAN communication device to transmit the tractor's steering angle control command. In addition, the tractor is equipped with a GPS that provides RTK level accuracy, and the driving path of the tractor is collected and used for performance verification.

Algorithm performance was estimated through the lateral error of the autonomous driving path from the reference path. The reference path information is the path that the passenger followed the ridge through direct driving, and the lateral error with the path information traveled through the proposed algorithm was calculated.

Table 2 shows the magnitude of the lateral error for each ridge type, and Figure 15 shows the lateral error of the sample data for each ridge type as a graph. Through this, it was confirmed that the average lateral error in all types of ridges was within the target value of 0.1 m. Also, the difference in the error size for each type is very small compared to the target average value. Through this, we verified that the performance of the algorithm proposed by different types of ridges is uniform.