The future of the agriculture industry is intertwined with automation. Accurate fruit detection, yield estimation, and harvest time estimation are crucial for optimizing agricultural practices. These tasks can be carried out by robots to reduce labour costs and improve the efficiency of the process. To do so, deep learning models should be trained to perform knowledge-based tasks. Which outlines the importance of contributing valuable data to the literature. In this paper, we introduce Raspberry PhenoSet, a phenology-based dataset designed for detecting and segmenting raspberry fruit across seven developmental stages. To the best of our knowledge, Raspberry PhenoSet is the first fruit dataset to integrate biology-based classification with fruit detection tasks, offering valuable insights for yield estimation and precise harvest timing. This dataset contains 1,869 high-resolution images, the highest quality in the literature, captured under controlled artificial lighting in a vertical farm. The dataset has a total of 6,968 instances of mask annotations, manually labelled to reflect the seven phenology stages. We have also benchmarked Raspberry PhenoSet using several state-of-the-art deep learning models, including YOLOv8, YOLOv10, RT-DETR, and Mask R-CNN, to provide a comprehensive evaluation of their performance on the dataset. Our results highlight the challenges of distinguishing subtle phenology stages and underscore the potential of Raspberry PhenoSet for both deep learning model development and practical robotic applications in agriculture, particularly in yield prediction and supply chain management. The dataset and the trained models are publicly available for future studies.

In this study, for the first time, a phenology-based dataset of raspberries was presented. Raspberry PhenoSet has several advantages that make it suitable for both academic research and industrial applications. The dataset has the highest resolution among previous fruit detection datasets, and it is also the first fruit dataset gathered in a vertical farming facility.

The dataset was gathered as the plants grew and underwent different phenology stages, namely, budding, flowering, and fruiting. Hence, it reflects various challenges present in a vertical farming facility, such as reflections from artificial lights, and occlusions caused by the random growth of stems and leaves. In Raspberry PhenoSet, the images were annotated based on the phenology stages, meaning that each class has a known number of days remaining before reaching its harvest time. This makes the dataset suitable for yield estimation and supply chain management.

State-of-the-art deep learning object detection and classification models were trained using the Raspberry PhenoSet and the results are provided as a baseline for future studies. Although the models have acceptable performance in detecting and classifying phenology stages, results indicate that even the top models struggle to distinguish some of the phenology stages. We believe this makes the Raspberry PhenoSet suitable for researchers developing deep-learning object detection and classification models.

If you have any questions, feel free to reach out to us at the following email us at:  removed for double-blind review