Date fruit dataset for intelligent harvesting

Research on automated date fruit harvesting is limited as there is no public dataset for date fruits to aid in this. Hence in this work, we built a comprehensive dataset for date fruits that can be used by the research community for multiple tasks including automated harvesting, visual yield estimation, and classification tasks. 

The dataset has been fully labeled, coded, and released with their associated files freely to the research community in the IEEE DataPort repository ( http://dx.doi.org/10.21227/x46j-sk98). 

• Advanced agriculture automation such as robotic harvesting can significantly increase quality and yield and reduce production costs and delay. This dataset is a contribution to agriculture automation research.

• The dataset advances the machine vision research of date fruit in pre-harvesting and harvesting stages. To the best of our knowledge, this is the only publicly available dataset for date fruit pre-harvesting and harvesting applications.

• The date dataset consists of images, videos, and weight measurements, which can be used for various machine vision applications for date fruit in orchard environments, including automated harvesting, fruit detection and segmentation, classification, maturity analysis, and weight/yield estimation.

• The dataset has multiple sets of variations that reflect the challenges in agriculture environments and date orchards, which is essential for building a reliable and robust machine vision.

The date fruit dataset was created to address the requirements of many applications in the pre-harvesting and harvesting stages. The two most important applications are automatic harvesting and visual yield estimation. Since date applications require different types of data with different characteristics, we built two separate datasets for the benefits of researchers in different date applications. Fig. 1 gives a brief overview of the two datasets.

The first dataset (dataset-1) contains 8079 images of date fruit bunches for five date varieties (Naboot Saif, Khalas, Barhi, Meneifi, and Sullaj), showing various pre-maturity and maturity stages. These images cover a large degree of variabilities in order to address the challenges in the date orchard environment, such as multi-scale images, variable illumination, different angles, and diverse bagging states. Dataset-1 can be used in many applications such as automatic harvesting tasks, fruit detection, fruit recognition, maturity analysis, etc.

The second dataset (dataset-2) consists of 152 Barhi date bunches belonging to 13 palms. All bunches were weighted after harvesting, and their images were captured in front of a white graph paper, as shown in  Fig. 6. Subset of these bunches (120 bunches from nine palms) were provided with more inclusive data: we marked the whole bunches in the palm, captured their images from different angles before and during harvesting, recorded 360° video for each palm, and registered their characteristics (height, trunk circumference, total yield, and number of bunches).

In addition to the whole bunch data, other information was collected: images and weights of the bunch components (individual dates and bunch stalk).

In this dataset, we arranged the data (dates, bunches, and palms and their related visual data, i.e., image/video files, or numerical data, e.g., weight), with a coding scheme to simplify referring, linking, and facilitating future extensions of the dataset. 

The coding scheme consists of four parts separated by dots, as shown in the following figure. The first part is the code for the palm tree, which consists of two parts: palm type (variety) and palm number. The palm number is omitted if the data is not associated with a specific palm (e.g. in the case of individual dates). The second part usage depends on the dataset. In dataset-2, this part consists of a character that refers to the maturity stage of the dates (I: Immature, K: Khalal, R: Rutab, and T: Tamar). In the dataset-1, this part refers to the imaging session code, which consists of a session symbol [S] and session number, e.g., S1 refers to the first imaging session. The imaging session was used in dataset-1 instead of referring to the maturity stage explicitly because this dataset was collected over several periods of time during the maturity development of the fruit. Therefore, in one period (imaging session) the dates may have multiple maturity stages (some dates ripen before others). The third part contains one or more symbols that refer to the type of data, as stated in the following figure. Multiple symbols in this part refer to multi-data, e.g., BW refers to a visual bunch data that also has a weight associated with it. The combination of these symbols should be in the same order as given in the following figure, starting with visual data symbols (B, S, T) followed by numerical data symbols (W, D). The fourth part consists of a numerical value that indicates the data sequence. In the single date images, where the image contains 16 dates, this part refers to the range sequence, e.g., 1-16.