We need to trace AgRP-expressing fibers from 30+ large images (~10,000 x 10,000 pixels) of tissue stained with DAB and photographed in a dark field.

Some preliminary work on segmenting fibers from images of immunoreactive tissue has shown us that we can get good segmentations training a network, like U-Net, with a single large image. It has also shown us that we can quickly generate a label by thresholding an image and manually removing white pixels that are not the desired class. Even though the threshold is not a perfect label, it was good enough to train a network to segment fibers. This is consistent with the literature; training with rough/lazy/partial labels can be enough to train segmentation networks [https://arxiv.org/abs/2101.03492, https://arxiv.org/abs/1906.12177]. In line with this idea, we can train a U-Net model with one image out of the 30+ and a rough label of that image to solve the fiber extraction for Vanessa's work.

An additional thought: We can evaluate the network on a second image, and if the segmentation is not as good as we hoped, we can retrain. This time, however, we would retrain with the prediction of the second image. We would correct the prediction in the same way we corrected the first rough label and retrain U-Net on this second image and its corrected prediction as its label. We could repeat this step until the network produces desired outputs. Given the preliminary results that showed U-Net performed well after training with just the thresholded image, we may not have to retrain on additional images.

We sample non-overlapping image patches of 256x256 with a training-to-validation ratio of 2:1.

To make the most of our single image, we can generate additional training samples by applying different transformations to copies of the image. In this case, we altered the brightness.

It is common wisdom that training a learning model with an unbalanced dataset can give you biased results. If you train a network to distinguish between cats and dogs with 99 images of cats and one of dogs, the network will learn to always predict "cat" regardless of input. Similarly, the proportion of empty image patches and image patches with fibers is large (i.e. 186 positive patches to 1564 negative patches) and could result in the network being biased to output empty predictions.

Here we leverage our augmentation step and generate new image patches in a proportion that will yield a final training set with the same number of negative and positive image patches. Our final training set contains 1564 positive patches and 1564 negative patches.

batch_size: 8

epochs: 151

use_channels: [0,1,2]

loss_opts:

name: Asymmetric Focal Loss

masked: True

weights: [0.2, 0.8]

label_smoothing: 0.1

model_opts:

args:

inchannels: 3

outchannels: 2

net_depth: 4

dropout: 0.2

spatial: True

first_channel_output: 16

output_act: 'softmax'

architecture: 'unet'

optim_opts:

name: "Adam"

args:

lr: 0.0001

We monitored the Intersection over Union metric during training to assess learning.

Since we are more interested in the path of a fiber than its contour, we can process the model outputs by passing them through a skeletonization process. We can then vectorize this skeleton and open it in a vector graphics software, like Adobe Illustrator.