Methodology

Image Dataset

To create a workflow that can discriminate, classify and count mosquito and non mosquito objects, it has been determined that 3 datasets are to be prepared for each of the main processes. These three datasets are to be used for supervised learning.

1. Object Detection of Mosquito and Non-mosquitos Objects

• This dataset will be used to train a Region Proposal Network that will help in isolating and differentiating objects of interest (foreground) from the rest of the image (background).

• This image dataset is to contain images of mosquitos and non mosquito objects scattered in variations of assortments. 

• The dataset will also contain annotations that provide the coordinates of each object and the correct number of objects in the image.

2. Image Discrimination of Mosquito and Non-mosquitos Images

• This dataset will be used to train a convolutional neural network that will determine if am single object is between two classes: mosquito, and non-mosquito. 

• This image dataset is to contain images of mosquitos and non-mosquito objects. 

3. Image Identification off Mosquito Species

• This dataset will be used to train a convolutional neural network that will determine if a mosquito object is of one of three classes: Aedes aegypti, Aedes albopictus, or Culex pipiens.

• This image dataset is to contain images of separate images of  Aedes aegypti, Aedes albopictus, or Culex pipiens.

For the Object Detection dataset, data is gathered via image acquisition of insect collection board setups before fogging and misting operations. The insect collection boards will be placed in specific areas indoors and outdoors. The setups are to be monitored and kept track before and after the fogging/misting operation. The images of setups before and after operations are to be collected as they are being monitored. The ideal image is to contain a collection of objects, including both mosquito and non-mosquitoes. 

Additional data will also be included to help train the model to recognize and differentiate foreground and background objects. This extra data is to include datasets where objects are manually placed into setups to help further train the model in its learning.

For the Discrimination dataset,  the image dataset is procured through a combination of manual image collection, and usage of existing image data available in the other datasets as well as online. Two classes are to be prepared, one for mosquito and the other for non-mosquito. For the non-mosuito image dataset,  image subjects are collected and are scrutinized to increase variety and help determine the characteristics that 

For the Classification Dataset,  the image dataset is procured through manually raising and photographing mosquitos.  The two classes, Aedes aegypti and Aedes albopictus are created from this. An additional  class, Culex pipiens is added through additional datasets of mosquitos available online. 

Pipeline Design

To create a streamlined pipeline, a proposed workflow making use of convolutional neural network models in a object detection algorithm is to be implemented. The convolutional networks will be made into separate modules to accomplish the respective objectives. In this implementation the models are to be stacked on each other, with the output of one being used as the input of the other model. The overall approach will be a modified approach to Faster R-CNN, which was introduced by Shaoqing Ren, et al. in 2015.

Following the Faster R-CNN approach, at the head of the pipeline, an object detection implementation is to be applied in order to isolate the mosquito and non-mosquito objects from the initial group image inputs. Once objects are localized, the image data of said objects are to go through image classifiers to be further classified into mosquito and non mosquito objects, and then lastly into a certain mosquito species.

Convolutional Neural Network

The workflow will be composed of neural networks, specifically a convolutional neural network (CNN). The CNN is a type of Deep Learning algorithm that is capable of taking in image inputs and successfully learns and extract features from image inputs, starting from low -level features to more complex high-level features. The images are translated into matrices which are to be fed into the model. Through this process, it can train a model to recognize nuances in patterns based on values and weights.  CNNs are most commonly used in image recognition tasks and other image related implementations. 

A binary classification approach was chosen to fit the needs of the proposed workflow. The break down of the discrimination and identification to be implemented can be broken down into a basic binary tree. The initial binary classification is for mosquito and non-mosquito, then if the object is classified as mosquito, it proceeds to the next classifier and so forth. 

During supervised training of each convolutional neural network, the model will be trained to learn to detect and classify on each level. Each classifier takes in an input csv file containing the bounding boxes and previous classifications to build upon it.   The output of the preceding step will become the input for the other. The output is updated to include the resulting class based on the classification as well as the confidence score. This follows a standardized format to allow for simplicity for the workflow

To determine the success of the training and tuning of the convolution neural network, the appropriate evaluation metrics were used to evaluate the image classification models.

Object Detection

The beginning of this proposed workflow begins with localizing and isolating objects of interest through object detection. The chosen method to achieve this is evaluated based on performance of determining and differentiating foreground from background via an object detection algorithm. Two methods were tested for comparison and for evaluation. The first approach makes uses of a two-stage RCNN Selective Search implementation, while the second one uses a one-stage Single Shot Detector from Tensorflow Object Detection API. 

The image dataset used to train train the foreground classifier was derived from the original object detection dataset. Using the proposed bounding boxes, it allows training using both positive (including objects) and negative data (excluding/obscured objects).

During the process of training the object detection model, an implementation of non-max suppression is used in the process.  Non-Max Suppression is an algorithm that is applied to systems involving region proposals to essentially filter out redundant output bounding boxes based on an objects Intersection over Union value. This is effective for single models and helps boost the precision overall.

Intersection over Union (for Results) - Average Precision (11-point interpolation) (for Results) - Pecision Recall Curve

After testing, the appropriate evaluation metrics for object detection algorithms were tested and recorded to ensure accuracy.

Once an image is successfully classified, a csv file for and output is produced. This output contains the bounding boxes of the objects of interest. This data will be used by the succeeding image classifier and will help discriminate and identify selected objects.

Mosquito Discrimination

The mosquito discrimination module for this pipeline will be built through the an implementation of an Image Classifier using a deep convolutional network.  The input object image will be classified between two classes: mosquito and non-mosquito. After classification, any classified mosquito objects are to be sent to the next step of the pipeline where the mosquito species is to be determined.

The MobilenetV2 architecture will be used for training the mosquito classifier. For training, a dataset containing mosquito and non-mosquito objects will be used as training data.

Mosquito Identification

Similar to the mosquito discrimination portion, the mosquito identification module for this pipeline will be built through the an implementation of an Image Classifier using a deep convolutional network. The input object image for this will come from confirmed mosquito objects and will be classified between three classes based on mosquito species: Aedes aegypti, Aedes albopictus, and Culex.

Mosquito Counting

For mosquito counting, mosquito successfully discriminated will be counted and kept track for the final tally after a run through of the pipeline.

Training

Dataset Preparation

To ensure a smooth and obstacle-free supervised training, certain processes were implemented before beginning training for each of the convolutional neural network models.  Datasets were prepared by being split into appropriate labeled folders for training, testing and validation. The data distribution between each followed a 8:1:1 ratio for the training, test, and validation sets respectively. Each of the dataset are properly labeled and kept separate with no overlap of image data. 

Data Augmentation was applied on the training image dataset before training begins to further minimize any additional computation needed. The image data was randomly adjusted per epoch of training. Changes included rotations, horizontal/vertical flips, brightness adjustments, hue adjustments, image zooms. These augmentations helped provide more variation as well as effectively increase the amount of training data without causing overfitting.

Model Training

Model training will be based on most optimal adjustments of hyperparameters for most efficient training run time and execution. Early stopping is also applied to prevent overfitting.  

Training is monitored and the results of the training will be logged and evaluated based on certain metrics.

Model Evaluation

The model trained for each of the main tasks, Object Detection, Mosquito Discrimination, and Mosquito Identification, will be evaluated based on performance metrics as well as other techniques. Evaluation metrics for image classification will be used such as accuracy, precision, recall, and F1 score.  Confusion Matrix and classification reports are to be tested and presented as well.

(To be updated)

Testing and Evaluation

Dataset Preparation

To ensure a smooth and obstacle-free supervised training, certain processes are to be implemented before beginning training each of the convolutional neural network models.  Datasets are prepared by being split into appropriate labeled folders for training, testing and validation. The data distribution between each follows a 8:1:1 ratio. Each of the datastet are porperly labeled and ketp separate. 

The training data will be used to 

Data Augmentations such as is to be applied on the training image dataset before training begins to further minimize any additional computation needed. The image data will be randomly adjusted per epoch (whole run though on image dataset). Changes may include rotations, horizontal/vertical flips, brightness adjustments, hue adjustments, image zooms. These augmentations will help provide more variation as well as effectively increase the amount of training data without causing overfitting.

Model Training

Model training will be based on most optimal adjustments of hyperparameters for most efficient training run time and execution. To determine the  most ideal setup for training, a number of combination of setups will be tested. 

• A selection of difference neural network architectures was tested to see which produces the best performing model of group

• Based on the best performing model of the group, it will be trained to see the best number of epochs for training.

Training is monitored and the results of the training will be logged and evaluated.

Model Evaluation

The model trained for each of the main tasks, Object Detection, Mosquito Discrimination, and Mosquito Identification, will be evaluated based on performance metrics as well as other techniques. Evaluation metrics for image classification will be used such as accuracy, precision, recall, and F1 score.  Confusion Matrix and classification reports are to be tested and presented as well.