Advanced Applied Deep Learning
Practice Course
Sheng Yun Wu
Practice Course
Sheng Yun Wu
In Week 8, students focus on training custom object detection models. This involves creating a custom dataset, labeling the dataset, configuring models like YOLO, SSD, or Faster R-CNN for training, and fine-tuning pre-trained models on custom datasets. The week provides hands-on experience in the end-to-end process of preparing data, training models, and evaluating performance.
Description:
This example introduces students to the process of creating a custom dataset for object detection. Students will capture or collect images and annotate them using tools like LabelImg to create XML or YOLO format annotations.
No Code for this example – Practical Explanation
Use LabelImg to manually annotate images.
Export the annotations in YOLO or PASCAL VOC format (depending on the model you plan to use).
Description:
Once the dataset is annotated, this example demonstrates how to prepare the dataset for training, including organizing images, annotations, and splitting the dataset into training and validation sets.
import os
import shutil
from sklearn.model_selection import train_test_split
# Define paths
image_dir = 'custom_dataset/images/'
label_dir = 'custom_dataset/labels/'
# Split dataset into train and validation sets
image_files = os.listdir(image_dir)
train_files, val_files = train_test_split(image_files, test_size=0.2, random_state=42)
# Move files to train and validation directories
os.makedirs('custom_dataset/train/images/', exist_ok=True)
os.makedirs('custom_dataset/train/labels/', exist_ok=True)
os.makedirs('custom_dataset/val/images/', exist_ok=True)
os.makedirs('custom_dataset/val/labels/', exist_ok=True)
for file in train_files:
shutil.move(image_dir + file, 'custom_dataset/train/images/')
shutil.move(label_dir + file.replace('.jpg', '.txt'), 'custom_dataset/train/labels/')
for file in val_files:
shutil.move(image_dir + file, 'custom_dataset/val/images/')
shutil.move(label_dir + file.replace('.jpg', '.txt'), 'custom_dataset/val/labels/')
Description:
This example shows how to configure YOLOv5 for training on custom data. This includes setting up the data YAML file, modifying model configurations, and preparing the model for training.
# data.yaml – Define custom dataset for YOLOv5
train: custom_dataset/train/images/
val: custom_dataset/val/images/
nc: 2 # Number of classes (e.g., 'dog', 'cat')
names: ['dog', 'cat'] # Class names
Code to Train:
# Train YOLOv5 on the custom dataset
!python train.py --img 640 --batch 16 --epochs 50 --data custom_dataset/data.yaml --weights yolov5s.pt
Description:
This example demonstrates how to fine-tune a pre-trained Faster R-CNN model on a custom dataset using PyTorch. The model is adjusted to detect custom objects.
import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision import transforms, datasets
from torch.utils.data import DataLoader
# Load the pre-trained Faster R-CNN model
model = fasterrcnn_resnet50_fpn(pretrained=True)
num_classes = 3 # Example: background + 2 classes
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = torch.nn.Linear(in_features, num_classes)
# Define dataset and dataloaders
transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.ImageFolder('custom_dataset/train/images/', transform=transform)
train_loader = DataLoader(train_dataset, batch_size=4, shuffle=True)
# Define optimizer
optimizer = torch.optim.SGD(model.parameters(), lr=0.005, momentum=0.9, weight_decay=0.0005)
# Fine-tune the model
model.train()
for epoch in range(10): # Example: 10 epochs
for images, targets in train_loader:
optimizer.zero_grad()
losses = model(images, targets)
loss = sum(loss for loss in losses.values())
loss.backward()
optimizer.step()
# Save the fine-tuned model
torch.save(model.state_dict(), 'faster_rcnn_custom.pth')
Description:
In this example, students learn how to train a pre-trained SSD model on a custom dataset using TensorFlow. This includes setting up the dataset and defining the model.
import tensorflow as tf
from object_detection.utils import config_util
from object_detection.builders import model_builder
# Load the pre-trained SSD model configuration
pipeline_config = 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config'
configs = config_util.get_configs_from_pipeline_file(pipeline_config)
model_config = configs['model']
# Build the model using the pre-trained SSD configuration
detection_model = model_builder.build(model_config=model_config, is_training=True)
# Define custom dataset for training (using TensorFlow's Object Detection API)
train_dataset = ... # Custom dataset loading logic
# Train the model
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
for epoch in range(50): # Example: 50 epochs
for images, targets in train_dataset:
with tf.GradientTape() as tape:
outputs = detection_model(images, training=True)
loss = compute_loss(outputs, targets) # Loss function
gradients = tape.gradient(loss, detection_model.trainable_variables)
optimizer.apply_gradients(zip(gradients, detection_model.trainable_variables))
# Save the trained SSD model
detection_model.save('ssd_custom_model')
Description:
Once the custom model is trained, this example demonstrates how to evaluate its performance on the validation set using mean Average Precision (mAP).
from sklearn.metrics import average_precision_score
# Assuming 'predictions' contains the predicted bounding boxes and 'true_labels' contains the ground truth
# Compute mean Average Precision (mAP)
def compute_map(true_labels, predictions):
all_precisions = []
for i, prediction in enumerate(predictions):
precision = average_precision_score(true_labels[i], prediction)
all_precisions.append(precision)
return sum(all_precisions) / len(all_precisions)
# Example: Compute mAP
map_value = compute_map(true_labels, predictions)
print(f"mAP: {map_value}")
Description:
This example demonstrates how to visualize the predictions from a trained custom model on new images to check the accuracy of bounding boxes and class labels.
import cv2
import matplotlib.pyplot as plt
# Load the trained model and the test image
model = torch.load('faster_rcnn_custom.pth')
image = cv2.imread('test_image.jpg')
# Perform inference
predictions = model(image)
# Draw bounding boxes on the image
for box in predictions['boxes']:
startX, startY, endX, endY = box
cv2.rectangle(image, (startX, startY), (endX, endY), (255, 0, 0), 2)
# Display the image with bounding boxes
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
plt.axis('off')
plt.show()
Description:
This example introduces how to tune hyperparameters, such as learning rate, batch size, and optimizer, to improve the performance of a custom object detection model.
# Hyperparameter tuning example: changing learning rates and batch sizes
learning_rates = [0.001, 0.0001]
batch_sizes = [8, 16]
for lr in learning_rates:
for batch_size in batch_sizes:
print(f"Training with learning rate: {lr}, batch size: {batch_size}")
# Compile and train model with custom learning rate and batch size
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=lr), loss='categorical_crossentropy')
model.fit(train_dataset, batch_size=batch_size, epochs=10)
Description:
This example helps students understand the importance of adjusting the number of training epochs based on the dataset and model performance.
# Fine-tune the number of epochs for training
for epochs in [10, 20, 30]:
print(f"Training for {epochs} epochs...")
model.fit(train_dataset, epochs=epochs, validation_data=val_dataset)
# Evaluate the model on the validation set
val_loss, val_acc = model.evaluate(val_dataset)
print(f"Validation accuracy after {epochs} epochs: {val_acc}")
Description:
This example shows how to export the trained custom model for deployment and how to use it for inference in a production environment.
# Export the trained YOLO model for deployment
!python export.py --weights 'best.pt' --img 640 --batch 16
# Use the exported model for inference
import torch
model = torch.hub.load('ultralytics/yolov5', 'custom', path='best.pt')
results = model('new_image.jpg')
# Display the results
results.show()
Objective: Train custom object detection models by fine-tuning pre-trained models (YOLO, Faster R-CNN, SSD) on a custom dataset.
Skills Developed:
Collect, annotate, and prepare a custom dataset for object detection.
Fine-tune pre-trained object detection models like YOLOv5, Faster R-CNN, and SSD.
Evaluate model performance using mean Average Precision (mAP) and visualize results.
Export and deploy trained object detection models.
Tools: TensorFlow, PyTorch, YOLO, OpenCV, LabelImg.
By the end of Week 8, students will be able to train their own object detection models on custom datasets, fine-tune hyperparameters, and evaluate model performance, gaining practical experience in building end-to-end object detection systems.