Getting Started with Image Segmentation

October 2022

https://courses.nvidia.com/courses/course-v1:DLI+L-FX-04+V2/

https://colab.research.google.com/drive/1YuQmW9SPLkn_zuvF2xopX4AvH2GdA2hq#scrollTo=C2k6Qi1dZNnL

Image classification - e.g. picture of a chair

Omage classification + localization - bounding box of object in image

Object detection - chain and bin

Image segmentation - pixel labelling

semantic segmentation - separate not by color but by class

Look at medical imaging and locate the left ventricle i.e. for each pixel is it part of the LV

Can make Keras models as sequential or functional

e.g.

model = tf.keras.models.Sequenctial([Flatten(input_shape=[256, 256, 1]), Dense(64, activation="relu"), Dense(256 * 256 * 2, activation="softmax"), Reshape((256, 256, 2))])

Prepare model - TFRecords

Build the model

Train the model

Event the model - DICE, tensorboard

Dataset

Cardiac MRI short-axis (SAX) scans

https://www.kaggle.com/competitions/second-annual-data-science-bowl

256 x 256 grayscale DICOM

output is 256 x 256 x 2 (LV or not)

234 images in training and validation of 26 images

images and contours have been extracted from the raw data for TF as TF data records

task 1

FC with one hidden layer (512) with sparse_softmax_cross_entropy_with_logits - softmax of the output then cross entropy against the correct labels - assign prob for each class and sorftmax so sum is = 1 - cross entropy means strongly punish wrong predictions if predictions 0% change but happens have a near infinite loss.

task 2

CNN layers - capture larger receptive fields. down-sampling method to retain info while eliminating some complexity

Transpose convolution - single pixel to multiple pixels.

max pooling - pick largest value

DICE metric - area of prediction area over area of actual value - perfect 1.0 - worse - 0.0

Task 3

experiment with learning_rate, decay_rate, decay_steps, num_epoch

Solution UNET

upscale stetch out the image

pairs conv going down with conv going up

each conv produces a new image. take image and stack with conv going up.