Advanced Applied Deep Learning

Practice Course

Sheng Yun Wu

Week 3: Transfer Learning and Pre-trained Models

In Week 3, the focus shifts to Transfer Learning, where students learn how to leverage pre-trained models to solve new problems with minimal training. This week involves fine-tuning popular architectures like VGG16, ResNet, and Inception, and understanding how transfer learning can accelerate the training process and improve performance on smaller datasets.

Example 1: Introduction to Transfer Learning

Description:
This example introduces the basic concept of transfer learning. It explains how a pre-trained model is used as a feature extractor by freezing the pre-trained layers and adding new custom layers for a new task.

from tensorflow.keras.applications import VGG16

# Load the pre-trained VGG16 model (without the top layers)

vgg_base = VGG16(weights='imagenet', include_top=False, input_shape=(150, 150, 3))

# Freeze the convolutional base

vgg_base.trainable = False

# Display the model architecture

vgg_base.summary()

Example 2: Fine-tuning VGG16 for Binary Classification

Description:
This example demonstrates how to fine-tune the pre-trained VGG16 model for a binary classification task, such as classifying images of cats and dogs.

import tensorflow_datasets as tfds

# Load the cats_vs_dogs dataset

(ds_train, ds_test), ds_info = tfds.load('cats_vs_dogs', split=['train[:80%]', 'train[80%:]'], shuffle_files=True, as_supervised=True, with_info=True)

# Display the dataset information

print(ds_info)

import os

import tensorflow as tf

from tensorflow.keras.preprocessing.image import ImageDataGenerator

from tensorflow.keras import layers, models

# Replace train_data and validation_data with the dataset which you downloaded.

base_path = r"C:\Users\asd01\tensorflow_datasets\cats_vs_dogs"

train_path = os.path.join(base_path , 'train')

validation_path = os.path.join(base_path , 'validation')

# Add custom layers to the pre-trained VGG16 base

model = models.Sequential([vgg_base, layers.Flatten(), layers.Dense(256, activation='relu'), layers.Dropout(0.5), layers.Dense(1, activation='sigmoid')])

# Compile the model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Normalized data

train_datagen = ImageDataGenerator(rescale=1./255)

validation_datagen = ImageDataGenerator(rescale=1./255)

# Adjust data

train_data = train_datagen.flow_from_directory(train_path, target_size=(150, 150), batch_size=16, class_mode='binary')

validation_data = validation_datagen.flow_from_directory(validation_path, target_size=(150, 150), batch_size=16, class_mode='binary')

# Train the model

model.fit(train_data, epochs=5, validation_data=validation_data)

Example 3: Loading and Using Pre-trained ResNet50

Description:
This example demonstrates how to use ResNet50, a deeper network architecture, for transfer learning. It shows how to load a pre-trained ResNet50 model and use it for feature extraction.

import tensorflow as tf

from tensorflow.keras.preprocessing.image import ImageDataGenerator

from tensorflow.keras import layers, models

# Add custom layers to the pre-trained VGG16 base

model = models.Sequential([

vgg_base,

layers.Flatten(),

layers.Dense(256, activation='relu'),

layers.Dropout(0.5),

layers.Dense(1, activation='sigmoid')

])

# Compile the model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Replace train_data and validation_data with your data

train_path = r"C:\Users\asd01\tensorflow_datasets\cats_vs_dogs\train"

validation_path = r"C:\Users\asd01\tensorflow_datasets\cats_vs_dogs\validation"

# Normalized data

train_datagen = ImageDataGenerator(rescale=1./255)

validation_datagen = ImageDataGenerator(rescale=1./255)

# Adjust data

train_data = train_datagen.flow_from_directory(

train_path,

target_size=(150, 150),

batch_size=16,

class_mode='binary'

)

validation_data = validation_datagen.flow_from_directory(

validation_path,

target_size=(150, 150),

batch_size=16,

class_mode='binary'

)

# Train the model

model.fit(train_data, epochs=5, validation_data=validation_data)

Example 4: Fine-tuning Pre-trained InceptionV3

Description:
This example shows how to fine-tune the InceptionV3 model, another popular pre-trained architecture, for a multi-class classification task.

from tensorflow.keras.applications import InceptionV3

# Load pre-trained InceptionV3 model (without top layers)

inception_base = InceptionV3(weights='imagenet', include_top=False, input_shape=(150, 150, 3))

# Freeze the convolutional base

inception_base.trainable = False

# Add custom layers for multi-class classification

model = models.Sequential([

inception_base,

layers.GlobalAveragePooling2D(),

layers.Dense(512, activation='relu'),

layers.Dropout(0.5),

layers.Dense(1, activation='sigmoid')

])

# Compile and train the model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model.fit(train_data, epochs=10, validation_data=validation_data)

Example 5: Unfreezing Layers for Fine-tuning

Description:
This example explains how to selectively unfreeze certain layers of the pre-trained model (such as the top few layers) to allow for fine-tuning.

# Unfreeze the last few layers of the convolutional base

for layer in vgg_base.layers[-4:]:

layer.trainable = True

# Compile the model again after modifying trainable layers

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Fine-tune the model

model.fit(train_data, epochs=5, validation_data=validation_data)

Example 6: Feature Extraction Using Pre-trained Models

Description:
In this example, students use a pre-trained model as a feature extractor. Instead of training the model end-to-end, the convolutional base is used to extract features, and a simple classifier is trained on top.

import numpy as np

from tensorflow.keras import optimizers

# Setting the samples number in train set and validation set (it dependon your samples)

num_train_samples = 19990

num_validation_samples = 4994

#setting parameters

image_size = 150

batch_size = 32

# Using ImageDataGenerator to pre-processing the data

datagen = ImageDataGenerator(rescale=1./255)

# Define the extract features function

# Extract features from the pre-trained VGG16 model

def extract_features(directory, sample_count):

features = np.zeros(shape=(sample_count, 4, 4, 512))

labels = np.zeros(shape=(sample_count))

generator = datagen.flow_from_directory(

directory,

target_size=(image_size, image_size),

batch_size=batch_size,

class_mode='binary',

shuffle=False)

i = 0

for inputs_batch, labels_batch in generator:

features_batch = vgg_base.predict(inputs_batch)

features[i * batch_size : (i + 1) * batch_size] = features_batch

labels[i * batch_size : (i + 1) * batch_size] = labels_batch

i += 1

if i * batch_size >= sample_count:

break

return features, labels

# Extract features from train set and validation set

train_features, train_labels = extract_features(train_dir, num_train_samples)

validation_features, validation_labels = extract_features(validation_dir, num_validation_samples)

# Flatten features

train_features = np.reshape(train_features, (num_train_samples, 4 * 4 * 512))

validation_features = np.reshape(validation_features, (num_validation_samples, 4 * 4 * 512))

# Build a new model on top of the extracted features

model = models.Sequential([layers.Dense(256, activation='relu', input_dim=4 * 4 * 512), layers.Dropout(0.5), layers.Dense(1, activation='sigmoid')])

# Compile and train the classifier

model.compile(optimizer=optimizers.RMSprop(learning_rate=2e-5), loss='binary_crossentropy', metrics=['accuracy'])

history = model.fit(train_features, train_labels, epochs=30, batch_size=batch_size, validation_data=(validation_features, validation_labels))

Example 7: Transfer Learning on Small Datasets

Description:
This example demonstrates how transfer learning can be particularly useful for small datasets where training a model from scratch would lead to overfitting.

# Load a small dataset (for demonstration, using a subset of a larger dataset)

def get_first_n_samples(iterator, n):

images_list = []

labels_list = []

samples_collected = 0

iterator.reset()

while samples_collected < n:

images_batch, labels_batch = next(iterator)

batch_size = images_batch.shape[0]

if samples_collected + batch_size > n:

excess = samples_collected + batch_size - n

images_batch = images_batch[:batch_size - excess]

labels_batch = labels_batch[:batch_size - excess]

images_list.append(images_batch)

labels_list.append(labels_batch)

samples_collected += images_batch.shape[0]

images_array = np.concatenate(images_list, axis=0)

labels_array = np.concatenate(labels_list, axis=0)

return images_array, labels_array

# Get the first hundred samples

small_train_data, small_train_labels = get_first_n_samples(train_data, 100)

# Fine-tune the pre-trained model on the small dataset

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model.fit(small_train_data, small_train_labels, epochs=5, validation_data=validation_data)

Example 8: Transfer Learning with Data Augmentation

Description:
Here, students combine transfer learning with data augmentation techniques to further improve model generalization.

# Build model

inputs = layers.Input(shape=(150, 150, 3))

x = vgg_base(inputs, training=False)

x = layers.Flatten()(x)

x = layers.Dense(256, activation='relu')(x)

x = layers.Dropout(0.5)(x)

outputs = layers.Dense(1, activation='sigmoid')(x)

model = models.Model(inputs, outputs)

# Apply data augmentation

train_datagen = ImageDataGenerator(

rescale=1./255,

rotation_range=40,

width_shift_range=0.2,

height_shift_range=0.2,

shear_range=0.2,

zoom_range=0.2,

horizontal_flip=True,

fill_mode='nearest'

)

# Data generator

validation_datagen = ImageDataGenerator(rescale=1./255)

# Build train data generator

train_generator = train_datagen.flow_from_directory(

train_path,

target_size=(150, 150),

batch_size=32,

class_mode='binary'

)

# Build validation data generator

validation_generator = validation_datagen.flow_from_directory(

validation_path,

target_size=(150, 150),

batch_size=32,

class_mode='binary'

)

# Compile the model

model.compile(optimizer=optimizers.Adam(),

loss='binary_crossentropy',

metrics=['accuracy'])

# Train the model

history = model.fit(

train_generator,

steps_per_epoch=train_generator.samples // train_generator.batch_size,

epochs=10,

validation_data=validation_generator,

validation_steps=validation_generator.samples // validation_generator.batch_size

)

Example 9: Transfer Learning with Custom Pre-trained Models

Description:
This example demonstrates how students can use their own pre-trained models for transfer learning by saving and loading models trained on different datasets.

# If you have your new data, you can replace the "train_data" and "validation_data" with your new data set in the new_model.fit

# Here, we just demonstrate how to use

# Save a model after training on a large dataset

model.save('custom_pretrained_model-dlc.h5')

# Load the pre-trained model

custom_model = models.load_model('custom_pretrained_model-dlc.h5')

# Use the custom model for transfer learning

custom_model.trainable = False

new_model = models.Sequential([

custom_model,

layers.Flatten(),

layers.Dense(128, activation='relu'),

layers.Dense(1, activation='sigmoid')

])

# If you have your new data, you can replace the "train_data" and "validation_data" with your new data set in the new_model.fit

# Here, we just demonstrate how to use

# Compile and train the model

new_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

new_model.fit(train_data, epochs=5, validation_data=validation_data)

Example 10: Evaluating Transfer Learning Performance

Description:
This example focuses on evaluating the performance of a transfer learning model by comparing it with models trained from scratch.

test_path = r"C:\Users\asd01\tensorflow_datasets\cats_vs_dogs\test"

# Build a test data generator

test_datagen = ImageDataGenerator(rescale=1./255)

test_data = test_datagen.flow_from_directory(

test_path,

target_size=(150, 150),

batch_size=batch_size,

class_mode='binary',

shuffle=False

)

# Evaluate the transfer learning model on the test set

test_loss, test_acc = model.evaluate(

test_data,

steps=test_data.samples // test_data.batch_size

)

print(f"Test accuracy with transfer learning: {test_acc}")

# Compare the accuracy with a model trained from scratch

new_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

new_model.fit(train_data, epochs=3, validation_data=validation_data)

scratch_loss, scratch_acc = new_model.evaluate(test_data)

print(f"Test accuracy without transfer learning: {scratch_acc}")

Week 3 Summary

Objective: Learn and implement transfer learning by using pre-trained models like VGG16, ResNet, and Inception for various classification tasks.
Skills Developed:
- Understanding transfer learning concepts, fine-tuning pre-trained models, extracting features, applying data augmentation with transfer learning.
- Gaining practical experience with pre-trained models for small datasets and fine-tuning techniques.
Tools: TensorFlow, Keras, Pre-trained models (VGG16, ResNet, InceptionV3).

These 10 examples in Week 3 cover the essential aspects of transfer learning, giving students hands-on experience with various pre-trained models and methods to fine-tune them for new tasks. This helps students understand how to leverage the power of large models while working with limited data and computational resources.

Page updated

Report abuse