Advanced Applied Deep Learning
Practice Course
Sheng Yun Wu
Practice Course
Sheng Yun Wu
In Week 9, students will explore transfer learning techniques for object detection. Transfer learning allows students to leverage pre-trained models to solve new object detection tasks with limited data. This week’s focus will be on fine-tuning popular pre-trained models (such as YOLO, Faster R-CNN, and SSD) for custom datasets and learning how to achieve better performance without training from scratch.
Description:
This example introduces the concept of transfer learning for object detection and explains the key steps involved: using pre-trained weights, replacing the classification head, and fine-tuning on a new dataset.
No Code for this example – Theoretical Explanation
Use pre-trained models like YOLO, Faster R-CNN, SSD.
Fine-tuning means modifying the classification layer and adjusting the training process for a specific task.
Description:
This example demonstrates how to use a pre-trained YOLOv5 model for transfer learning by fine-tuning it on a custom dataset.
# Assuming custom data is prepared and the data.yaml file is configured
# Fine-tune pre-trained YOLOv5 model on custom data
!python train.py --img 640 --batch 16 --epochs 30 --data custom_dataset/data.yaml --weights yolov5s.pt
Description:
In this example, students fine-tune a pre-trained Faster R-CNN model using PyTorch’s torchvision library on a custom dataset with new object classes.
import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision import transforms, datasets
from torch.utils.data import DataLoader
# Load pre-trained Faster R-CNN model
model = fasterrcnn_resnet50_fpn(pretrained=True)
num_classes = 3 # Modify based on the number of custom classes
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = torch.nn.Linear(in_features, num_classes)
# Load custom dataset
transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.ImageFolder('custom_dataset/train/images/', transform=transform)
train_loader = DataLoader(train_dataset, batch_size=4, shuffle=True)
# Fine-tune the model
optimizer = torch.optim.SGD(model.parameters(), lr=0.005, momentum=0.9, weight_decay=0.0005)
model.train()
for epoch in range(5): # Fine-tune for 5 epochs
for images, targets in train_loader:
optimizer.zero_grad()
loss_dict = model(images, targets)
losses = sum(loss for loss in loss_dict.values())
losses.backward()
optimizer.step()
# Save the fine-tuned model
torch.save(model.state_dict(), 'faster_rcnn_custom_finetuned.pth')
Description:
This example shows how to fine-tune a pre-trained SSD model using TensorFlow’s Object Detection API for a custom dataset.
import tensorflow as tf
from object_detection.utils import config_util
from object_detection.builders import model_builder
# Load the pre-trained SSD model configuration
pipeline_config = 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config'
configs = config_util.get_configs_from_pipeline_file(pipeline_config)
model_config = configs['model']
# Build the model
detection_model = model_builder.build(model_config=model_config, is_training=True)
# Load custom dataset
train_dataset = ... # Custom dataset loading logic
# Fine-tune the model
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
for epoch in range(10): # Example: 10 epochs
for images, targets in train_dataset:
with tf.GradientTape() as tape:
outputs = detection_model(images, training=True)
loss = compute_loss(outputs, targets) # Custom loss function
gradients = tape.gradient(loss, detection_model.trainable_variables)
optimizer.apply_gradients(zip(gradients, detection_model.trainable_variables))
# Save the fine-tuned model
detection_model.save('ssd_finetuned_custom')
Description:
In this example, students will learn how to fine-tune the YOLOv4 model on a custom dataset for transfer learning.
# Fine-tune YOLOv4 model on custom dataset
!darknet detector train custom_dataset/data.data cfg/yolov4.cfg yolov4.conv.137
Description:
This example shows how to evaluate the fine-tuned YOLOv5 model on a validation set and calculate the model’s accuracy and mean Average Precision (mAP).
# Evaluate the YOLOv5 model
!python val.py --data custom_dataset/data.yaml --weights runs/train/exp/weights/best.pt --img 640
Description:
In this example, students evaluate the performance of a fine-tuned Faster R-CNN model on a custom dataset using metrics like mean Average Precision (mAP).
from torchmetrics.detection.mean_ap import MeanAveragePrecision
import torch
# Define mAP metric
metric = MeanAveragePrecision()
# Load test data and perform inference
test_dataset = datasets.ImageFolder('custom_dataset/val/images/', transform=transform)
test_loader = DataLoader(test_dataset, batch_size=4, shuffle=False)
model.eval()
# Evaluate the model on test data
for images, targets in test_loader:
outputs = model(images)
metric.update(outputs, targets)
# Calculate mAP
map_value = metric.compute()
print(f"Mean Average Precision: {map_value}")
Description:
This example demonstrates how to fine-tune hyperparameters like learning rate, batch size, and number of epochs to improve the performance of the fine-tuned model.
# Fine-tune learning rate and batch size
learning_rates = [0.001, 0.0001]
batch_sizes = [8, 16]
for lr in learning_rates:
for batch_size in batch_sizes:
print(f"Fine-tuning with learning rate: {lr}, batch size: {batch_size}")
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=lr), loss='categorical_crossentropy')
model.fit(train_dataset, batch_size=batch_size, epochs=10)
Description:
This example demonstrates how to visualize the results of the fine-tuned object detection model on new images by drawing bounding boxes around detected objects.
import cv2
import matplotlib.pyplot as plt
# Load fine-tuned YOLOv5 model
model = torch.hub.load('ultralytics/yolov5', 'custom', path='runs/train/exp/weights/best.pt')
# Perform inference on a new image
results = model('new_image.jpg')
# Visualize the results
results.show()
# Save the image with bounding boxes
results.save()
Description:
This example shows how to export the fine-tuned object detection model and deploy it for real-time inference.
# Export the fine-tuned YOLOv5 model
!python export.py --weights 'runs/train/exp/weights/best.pt' --img 640 --batch 16
# Use the exported model for real-time inference
import torch
model = torch.hub.load('ultralytics/yolov5', 'custom', path='best.pt')
# Perform inference on a video stream (e.g., webcam)
cap = cv2.VideoCapture(0)
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
results = model(frame)
results.show() # Display real-time detection
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Objective: Leverage transfer learning techniques to fine-tune pre-trained object detection models on custom datasets.
Skills Developed:
Fine-tune popular object detection models (YOLOv5, Faster R-CNN, SSD) using transfer learning.
Evaluate fine-tuned models on custom datasets using metrics like mAP.
Hyperparameter tuning for improved performance in transfer learning scenarios.
Visualize and deploy fine-tuned object detection models for real-time inference.
Tools: PyTorch, TensorFlow, YOLOv5, YOLOv4, OpenCV.
By the end of Week 9, students will have gained practical experience with transfer learning, fine-tuning pre-trained models for new object detection tasks, and deploying these models for real-time applications.