Evaluation and Testing

Firstly, we will test the system with some z-stacks of the same cell types (MDA-MB-231 Cells) multiple times to see if the system can accurately identify and label cancerous cells. If the system targets the correct cell phenotypes, then it is acceptable for the lab purpose. This can be done for multiple different z-stacks to find the robustness for the system for this cell type. We determined the number of spherical objects, non-spherical objects, and masks detected by the algorithm. The standard for total number of spherical objects was defined as what was determined by a human. The true positive, false positive, and false-negative rates will be measured in order to determine the accuracy of the algorithm (which can be visualized above). Using these measurements, the sensitivity of the algorithm was then determined. By conducting experimentation on the number of false positives and the accuracy of the system, one could predict, given a certain number of images, how many of the cancer cells the software will identify and how many times it will falsely label a non-cancerous cell or any other feature in the gel.

The algorithm used a variety of tests to measure circularity. A consequence of this is that categorized cell populations on the edge of the z-stack image may have been incorrectly identified as circular. The algorithm divided objects into 3 categories: spherical, non-spherical, and masks. The masked group was defined as the number of ideal targets that are spherical. An ideal target would be an object that is not on the edge of the image and has no objects above or below it. The accuracy of the algorithm would be determined based on its ability to correctly select objects from the mask group. In this task, the detector achieved a sensitivity of 55.2% with 0.099±0.03 false positives per image (see table 1). Furthermore, the algorithm also had an accuracy of 57.1% (see table 1). The results of this experiment serve as a proof of concept to show that the algorithm can successfully identify and sort cell masses in the 3D cell culture environment.

The above figure shows the false positive, false, negative, and true positive rates of our phenotypic detection algorithm. One can observe that the algorithm was able to obtain a high rate of detection for true positives (0.132± 0.04) while simultaneously having a low false-positive rate (0.099 ± 0.03). As the algorithm eliminated non-ideal candidates, there was also a high false-negative rate (0.11 ± 0.03).

We also determined the amount of time it takes for the algorithm to finish running. This will be compared to the time it takes for an experimenter to photoconvert all the cells in a single collagen gel. A run time test would also be necessary to determine how much time the algorithm can save for the researcher. On average, it takes approximately 2 hours for a human to manually identify the cancerous MDA-MB-231 cells. Our algorithm can create a set of z-stack images in 11 minutes (660 seconds) and accomplishes the task of automatic identification of the cancerous cells in approximately 55 seconds, yielding a total time of 715 seconds. As a result, we obtain an improvement of 90.07% (see table 2).