GSAFE

WHAT IS PROJECT GSAFE?

GSAFE (Gemma Smart Analysis for Fungus Entities) is a tool developed for checking the edibility of mushrooms using the Google Gemma 3n model. The main reasons for the choice of Gemma 3n for this project are:

• it is a generative AI light model, which can run on devices such as smartphones

• it supports multimodal input, i.e., text, audio, video

• it can be fine-tuned, e.g. by using Unsloth.

WHAT MOTIVATED GSAFE?

Mushrooms can be poisonous, and knowing when they are safe is not common knowledge. Picking them up to check later is not a sustainable strategy. Real-time online check is also sometimes not possible because Internet connection may not be available in locations where mushrooms grow. GSAFE can run locally on a smartphone and it does not require Internet connection to work.

Sometimes I also see mushrooms in Singapore, and I am not sure if they are good or not. You can have a look at some of the mushrooms I found in Singapore here (spoiler: they are not safe, as confirmed later by GSAFE).

This is also a project for the Google Gemma 3n Impact Challenge hackatlon, hosted by Kaggle.

HOW WAS GSAFE DESIGNED?

The native Gemma 3n model is not designed to identify mushrooms. A first option considered was to train a CNN to complement the model, but this would have been more computationally challenging to run and the multimodal capabilities of Gemma 3n would not have been fully utilised. Therefore, it was discarded.

A second approach that we tested was fine-tuning using Unsloth with LoRA, and a public database of mushrooms classified as edible or poisonous. The resulting model was less conservative (less mushrooms labelled as poisonous), but also less safe as more poisonous mushrooms were classified as edible (see below). In addition, there were some technical challenges in the fine-tuning and loading the resulting model for inference. Therefore, this was not the approach selected for GSAFE. Nevertheless, you can find:

• the fine-tuning procedure here

• the resulting fine-tuned model here.

Eventually, the strategy selected for GSAFE was to employ some specific prompts to make it work as an expert in mushroom classification. You can check the writeup of the project here.

HOW DOES GSAFE WORK?

You take a picture of mushrooms and GSAFE will tell you if poisonous or edible. That's it. 

If you want to see how it looks like, you can check testing examples here, or the summary video below.

PROS, CONS, AND FUTURE WORK

✅ GSAFE can help identify mushrooms using devices such as smartphones in locations where Internet connection is not available. Therefore, there is no need to pick unsafe mushrooms only to discard them later and damage their native environment. This mitigates the health risks of eating poisonous mushrooms. 

🆘 Poisonous mushrooms can still be misclassified. Therefore, it is important to collect more data (labelled pictures of mushrooms) to improve fine-tuning performance.

💡The next step after fine-tuning results are satisfactory is to develop a tool that can be shared. You can see a first example below, made with Gradio. A good feature would be to allow users uploading their pictures with correct labels to improve performance (the model is likely to be used only for a few types of mushrooms depending on user location).

If you have ideas, comments, suggestions, feel free to get in touch!

RESOURCES

• Fine-tuning procedure with Unsloth (Kaggle Notebook) ▶️here

• Fine-tuned models ▶️here

• Public mushrooms dataset used for fine-tuning ▶️here

• Testing experiments (Kaggle Notebook) ▶️here

• The Kaggle Writeup of the project ▶️here