The purpose of this work was to develop a cell detection program for future use. Specifically, we trained the deep learning model YOLOv5 on an existing dataset and deployed it within a graphical user interface. The end product is a custom-built tool to facilitate the collection of similar data for future experiments. This work will be presented as a poster at the Society for Neuroscience conference in the fall of 2022.

We include the abstract of the poster below for context, but the remainder of this post expands on the implementation of the tool. We discuss how we developed the main component, the object detector, and then explain the logic behind the graphical user interface.

Subgroups of hindbrain neuronal populations are well documented to respond robustly to glycemic challenge. These populations, however, lay scattered within a vast expanse of neural tissue that requires time, labor, and expertise to analyze quantitatively. To reduce this labor but maintain quality control from the expertise, we developed an algorithm that employs the deep learning model, YOLOv5, to quickly and accurately annotate cells in epifluorescence photomicrographs. These cells include those that displayed rapid activation in association with an intravenous 2-deoxy-glucose challenge (250 mg/kg) relative to a saline-treated control group. The cells were identified using dual and/or triple-label immunohistochemistry for dopamine beta hydroxylase, choline acetyltransferase, and the cellular activation marker, phosphorylated-ERK1/2. The model was trained in-house on a subset of manual cell markings from both treatment groups in the locus ceruleus, nucleus of the solitary tract, and dorsal motor nucleus of the vagus to detect cellular positions. We reviewed the preliminary cell markings and manually adjusted the model predictions to remove false-positives and register cells that may have gone undetected by the program. The precision and recall of the model were evaluated so that it could reliably serve as a first-pass cell-locator for the reviewer of the data. This method will allow us to streamline our future data collection and analysis to map hindbrain chemoarchitecture and activated neuronal populations to an open-access rat brain atlas.

YOLOv5 requires a specific format of labels for objects. Namely, for every object, YOLO requires the class, the x, and y center coordinates and the width and height, in that order. The labels take the form of a plain text file, where each line represents an object and a single text file contains all objects in an image. An example is shown below (borrowed from here). Note x, y, width, and height are normalized to the width and height of the image.

Now that we had labels in the proper format (either fixed or learned), we proceeded to prepare the data for the main task. We randomly separated our main dataset into a training and validation set, using the same 2:1 ratio. This separated our 143 images into 95 image-SVG pairs for training and 48 image-SVG pairs for validating. For all images, we sampled crops of 256x256 pixels. Then, to ensure a balanced dataset, we enforced certain conditions while sampling positive and negative samples.

We considered crops that contained annotated cells in them as positive samples. These were obtained by iterating through the images in a grid-like fashion and sampling all non-overlapping crops of 256x256 that contained labeled cells. Then, for negative samples, we couldn't just sample crops without annotated cells because our dataset is not fully labeled, meaning we could risk sampling a "negative" crop when in fact it has cells that have not been labeled. To overcome this, we drew polygons around all cells (labeled or unlabeled) on all 143 images. Then during the sampling, we would check if the crop contained the polygons; if it didn't we would sample it as a negative sample, otherwise, we'd skip it. Lastly, for each image, we sampled as many negative samples as there were positive samples, thus ensuring a balanced dataset. The figure below illustrates the sampling of negative and positive samples, in red and green respectively, including the cell annotations in green and the negative sampling regions overlayed with gray lines.

We used the YOLOv5 implementation developed by Ultralytics. We tested 200 configurations, each with different image augmentations (and different strengths), model configurations, batch sizes, and optimizers. We ran these experiments on four NVIDIA GeForce RTX 2080 Ti graphics cards for a total duration of 20 hours. From all these, the best instance used Ultralytics' default augmentation parameters in "low-augmentations", the YOLOv5s (small) model configuration, a batch size of 32 images, and the Adam optimizer with an initial learning rate of 0.001. All other training parameters remained as default.

Here we show the effects of the image augmentations performed during training. These are the augmentations that resulted in the best-performing model. Each square here represents an input image of size 256x256. At the same time, here show the difference in bounding boxes. On the left, we can see what labels (red boxes) look like with the fixed bounding squares, and on the right, we show the learned bounding boxes.

Here we present the final training results of training with fixed bounding squares and learned bounding boxes. It is evident that doing the additional step of generating bounding boxes that are closer to the cell anatomy improves results in the main task of detecting cells.

During inference time, the model provides a level of confidence for every bounding box prediction. This value tells you how confident the model is that a cell actually exists inside the bounding box. This can be a useful parameter when using the model in the deployment phase since it allows you to filter predictions based on a confidence level. Typically, the F1 Score is used to find the optimal confidence level since it measures the trade-off between false positives and false negatives. A higher F1 Score is better, and we can plot the F1 score versus the confidence level to reveal which confidence level yields the highest F1 score.

Here we show some sample predictions side-by-side with the human label for comparison.

Now that we have a trained network, we can use it to extract preliminary cell counts in future experiments. Here we elaborate on the graphical user interface its features. We also explain an additional process in the pipeline that facilitates Colocalization Analysis.

For ease of use, we released the model in a web app graphical user interface. The web app can be downloaded from GitHub and installed on a personal computer and the code can be found here. It only uses the browser for interface purposes, but it does not require an internet connection and all files remain local.

If colocalization analysis is enabled, the program assumes the images are aligned and share the same dimensions. The program will run the object detection algorithm on each image, and then use the detections to run a colocalization analysis for all possible combinations. The program will then output an SVG for each image and each combination.

As an example, to run the colocalization of peptides pERK, ChAT, and DBH, the user would need to input an image for each peptide. The program will run the object detection algorithm on each image, and then use the detections to run colocalization analysis for pERK-ChAT, pERK-DBH, ChAT-DBH, and pERK-ChAT-DBH. The program will then output a total of seven SVG files.