Tomato Ripeness Detection Using ESP32-CAM and Machine Learning

This project developed a practical, low-cost Edge AI solution for real-time tomato ripeness detection using a quantized MobileNetV2 model deployed on the ESP32-CAM. The system operates entirely offline to deliver accurate, field-ready classification (92% average accuracy), democratizing advanced technology for farmers at all scales.

Problem Statement

The primary challenge this project addresses is the inefficiency and inaccessibility of modern agricultural automation for small to medium-scale farmers. This issue is multifaceted, hindering the widespread adoption of technologies that could significantly enhance productivity and reduce waste. From an engineering perspective, the core problems manifest as a set of design constraints:

• Operational Inefficiency: Manual inspection is a low-throughput, high-latency process that creates bottlenecks during critical harvest periods.

• Human Error and Inconsistency: Subjective human judgment leads to classification errors and inconsistent quality grading. This variability directly impacts market value, consumer satisfaction, and post-harvest losses.

• Connectivity Barriers: Many existing AI solutions are cloud-based, depending on stable, high-speed internet connectivity. Such infrastructure is often unavailable or unreliable in rural agricultural settings, rendering these solutions impractical for on-field deployment.

To address these constraints, we architected a fully-integrated, edge-first system designed for high performance on resource-constrained hardware.

Block Diagram

Methodology and Working

The methodology for the tomato project was structured to ensure accuracy, affordability, and edge deployability. The system's working follows a robust, entirely offline edge AI pipeline implemented on the ESP32-CAM.

Methodology

The process began with compiling a diverse dataset of over 3,000 annotated images, intentionally incorporating varied lighting, angles, and backgrounds to ensure real-world robustness. MobileNetV2 was selected as the foundational machine learning model due to its efficiency and use of depthwise separable convolutions, making it uniquely suited for low-power devices.

To achieve microcontroller compatibility, the trained Keras model was converted to the TFLite format and subjected to INT8 Post-Training Quantization. This crucial step converted 32-bit floating-point weights to 8-bit integers, reducing the model size from approximately 3.5 MB to ~500 KB (an 85% reduction) and making inference four times faster. The quantized TFLite model was then transformed into a C header array and compiled into the ESP32 firmware using TensorFlow Lite for Microcontrollers.

Working (System Pipeline)

The final system operates in three main stages:

1. Image Capture: The ESP32-CAM module, utilizing its OV2640 sensor, acquires high-resolution frames. These frames are internally downscaled to 224x224 to match the optimized MobileNetV2 input size.

2. AI Processing: The dedicated inference task loads the integer-only quantized model into the ESP32's statically allocated tensor arena. The model performs real-time classification, outputting a probability distribution over the three ripeness categories: Green, Yellow, and Red. This inference process typically takes between 350–700 ms per frame, enabling 2–3 frames per second performance.

3. Result Display: The predicted class label and its confidence score are immediately displayed to the user via an OLED interface using I²C communication, completing the system entirely on-device without any reliance on network connectivity.

Results and Discussion

The performance gap between the 95-96% accuracy in the lab and the 85-90% accuracy in the field is expected and demonstrates strong real-world generalization. The model maintains high reliability despite environmental noise like variable lighting, shadows, and complex backgrounds. In terms of speed, the system achieves an end-to-end latency of 350-700 ms per frame, which translates to a throughput of 2-3 frames per second. This throughput confirms the system's suitability for human-in-the-loop tasks such as manual sorting and mobile inspection, where real-time feedback is critical but sub-30 FPS performance is acceptable.

The project also aligns with several United Nations Sustainable Development Goals (UNSDGs) by creating a tool that promotes food security and technological innovation:

• SDG 2: Zero Hunger: By enabling more precise harvest timing, the system helps reduce post-harvest food losses, increasing the efficiency of food production.

• SDG 9: Industry, Innovation, and Infrastructure: The project represents an innovative application of edge AI and IoT, creating a low-cost, scalable automation tool that strengthens the technological capabilities of the agricultural industry.

• SDG 12: Responsible Consumption and Production: By minimizing food spoilage through optimal harvest scheduling, the system promotes more sustainable production patterns.

The empirical results and SDG alignment confirm that the system is a technically sound and practically deployable solution for low-cost agricultural automation.

Conclusion

This project successfully developed and validated a fully functional, real-time tomato ripeness detection system on an ESP32-CAM microcontroller. It demonstrates the technical feasibility of deploying sophisticated AI models on low-cost, resource-constrained hardware for practical agricultural applications. The primary outcomes are threefold: the project established that edge AI is a viable solution for field-based analysis, it delivered a cost-effective automation tool with a hardware budget under 3000 Rs, and it validated the system's performance with accuracy rates of 95-96% in controlled lab settings and 85-90% under variable field conditions. This work makes a meaningful contribution toward making sustainable, embedded AI accessible to the agricultural sector, empowering farmers with tools that enhance efficiency and reduce waste.

Future Scope

While the current system is a robust and field-ready prototype, several opportunities exist to enhance its capabilities and broaden its impact.

1. Multi-Fruit Recognition: The underlying model can be extended using transfer learning to classify the ripeness of other crops, such as peppers, citrus fruits, and strawberries, creating a versatile agricultural inspection tool.

2. Environmental Sensor Fusion: Integrating sensors for temperature, humidity, and soil moisture can provide additional environmental context, allowing for the development of predictive models to estimate ripening timelines.

3. Mobile and Web-Based Monitoring: A user-friendly mobile app or web dashboard could be developed to display real-time results, track historical data, and provide farm managers with a centralized interface for remote monitoring.

4. Hardware Optimization: Future hardware revisions could include a robust, waterproof enclosure and solar power integration, enhancing the system's durability for continuous, long-term outdoor deployment.