Project Proposal

Mid-term Report

https://docs.google.com/document/d/1bVaXncT92iR8GbGMuT09GvqUtNx6gY1CDO8NDBfLzfE/edit?usp=sharing

The Team

Full Cranberry Implementation

For the full cranberry case, our first thought was to use simple contour detection, utilizing the built in Python and OpenCV functions. One of the biggest problems we ran into early on was that some cranberries were colored too similarly to the surface the image was taken on, and the contours were not forming properly, if at all. We spent a long time brainstorming and implementing different image processing techniques to try and solve this issue. Some of these techniques included fine tuning blurs, changing the saturation, color thresholding, and and even more involved methods of semantic segmentation like a neural network to name a few.

In the end, we found that a simple canny edge detection after a Kuwahara filter worked very well. The Kuwahara filter blurs while preserving edges, which was perfect as we could blur the color variances within the cranberry so they wouldn't be mistaken as edges, while still preserving the important edges -- the actual edges of the cranberry.

Some of the contours generated using this method would still not create a closed figure however, disturbing our area calculations. In those cases, we simply closed the contours using a convex hull approximation. Using simple logic, we were able to differentiate cranberry contours from the objects of known size based on area, as well as ignoring smaller spurious contours detected as a result of noise. From there, we used the built in contourArea() function to get the area of each contour.

Locule Analysis Implementation

For the locule analysis case, we used Mathematica. There are a few different reasons for using Mathematica's built-in functions rather than something like OpenCV:

• Mathematica is very powerful prototyping playground with lots of built in Image Processing tools. It is where we did a lot of brainstorming.

• We have a working prototype in Mathematica, and the effort to figure out how to convert it to OpenCV looked to be more work than originally thought and planned for.

• It was working better in Mathematica than OpenCV, and Mathematica functions were more different compared to OpenCV than expected. For example, increasing the contrast of the images was not working the same way, and there looked to be extra work to re-implement the Watershed Segmentation method than we had originally thought.

• We wanted to prioritize having many different data points recorded with many different computer vision algorithms rather than putting in the work to re-implement computer vision algorithms ourselves.

• Mathematica is more user-friendly to those unfamiliar with code, such as the cranberry genetics lab faculty.

• We are still using applications of computer vision and image processing techniques and intuition learned from class, even though we are not re-implementing the fine details.

• We wanted to prioritize getting working end results by using already implemented image processing techniques than to put more time into re-implementing something that was not built into OpenCV.

1. Original - This is a cropped version of the original photo

2. Increase Contrast - Increasing the contrast by an absurd amount helps a lot to better see the locules, because they are very similar to the rest of the color of the cranberry. This makes the edges sharper.

3. Kuwahara Filter - This smooths the image while preserving edges. This helps to eliminate unwanted regions in the segmentation step

4. Watershed Segmentation - This step does the majority of the work, and takes the most amount of run time (about a minute or two). The Watershed Segmentation method was done many different times with different settings, because in experimentation, different settings worked better for different parts of the locules and different cranberries. The results of each were binarized and added together to get the final Locule Mask. The image shown is one of many instances, so that is why it does not quite look like it belongs with the rest of the steps.

   1. Separate the image from the previous step (Kuwahara filter) into it's RGB channels

   2. For each RGB channel, Binarize the image at different threshold values (from 0 to 1 with increases of 0.05)

   3. For each binary image from the previous step, we perform Mathematica's implementation of Watershed Segmentation algorithm with MinimumSaliency of 0.3 (this affects the "tolerance" of different smaller segments being considered to be combined into one larger segment)

5. Binarize to get Locule Mask - All results from each Watershed pass are visualized into a binary image and added together to get a resulting mask. Mathematica's DeleteSmallComponents function (which uses "cluster variance maximization (Otsu's algorithm)") is ran to remove random flecks still remaining in the image.

6. Contours of Original - Given original image again, returns a binary image of where there are zeros (black background)

7. Add Locule Mask to Contours of Original

8. Invert to get mask of the cranberries' "footprint" without the hollow locule

Full Cranberry Contour Analysis Output

Below is an example of visualizing the results right on the original image itself.

Below is just labeling the cranberries ID numbers so that the data can instead be placed into a CSV file where it can be more easily analyzed.

Below is a snapshot of an example CSV file produced from the analyzed image above.

The code also outputs the images of the cranberries individually - with their given ID number and also annotated versions.

Locule Analysis Output

Used mathematica to create a visual matrix of information about each cranberry. The format is: "Original", "Avg. Color", "Avg. Color (RGB 0-1)", "Mask", "Total area", "Hollow area", "Non-hollow area".

We also create a CSV file output of the same information in the same order, but without the image files (because those do not show up in a CSV file).