Advanced Applied Deep Learning
Practice Course
Sheng Yun Wu
Embedded Files
Week 2: Convolutional Neural Networks (CNNs) – Architecture and Layers
Week 2: Convolutional Neural Networks (CNNs) – Architecture and Layers
In Week 2, the focus is on building a deeper understanding of Convolutional Neural Networks (CNNs), their architecture, and key layers. This includes learning about convolutional layers, pooling layers, activation functions, and how they come together to create a CNN model for image classification. The examples are designed to progressively build the student's skills by implementing CNNs from scratch and visualizing key components.
Example 1: Understanding Convolutional Layers
Example 1: Understanding Convolutional Layers
Description:
This example introduces the concept of convolutional layers in CNNs. The code demonstrates how convolutional layers are used to extract features from images.
import tensorflow as tf
from tensorflow.keras import layers
# Input image (randomly generated for demonstration)
input_image = tf.random.normal([1, 28, 28, 1])
# Apply a convolutional layer
conv_layer = layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1))
output = conv_layer(input_image)
# Show the shape of the output
print(f"Output shape after convolution: {output.shape}")
Output
Output shape after convolution: (1, 26, 26, 32)
Example 2: Understanding Pooling Layers
Example 2: Understanding Pooling Layers
Description:
Students learn about max pooling layers and how they reduce the spatial dimensions of the feature maps while retaining important features.
# Apply a max pooling layer
pooling_layer = layers.MaxPooling2D((2, 2))
pooled_output = pooling_layer(output)
# Show the shape of the output after pooling
print(f"Output shape after pooling: {pooled_output.shape}")
Output
Output shape after pooling: (1, 13, 13, 32)
Example 3: Building a Simple CNN for Image Classification
Example 3: Building a Simple CNN for Image Classification
Description:
In this example, students build their first CNN for image classification using the MNIST dataset.
from tensorflow.keras import models
# Load MNIST dataset
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()
# Reshape and normalize the data
train_images = train_images.reshape((60000, 28, 28, 1)).astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1)).astype('float32') / 255
# Build a CNN model
cnn_model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the CNN
cnn_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 14s 7ms/step - loss: 0.1465 - accuracy: 0.9555 - val_loss: 0.0654 - val_accuracy: 0.9797
Epoch 2/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.0492 - accuracy: 0.9848 - val_loss: 0.0421 - val_accuracy: 0.9859
Epoch 3/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.0361 - accuracy: 0.9887 - val_loss: 0.0320 - val_accuracy: 0.9888
Epoch 4/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.0270 - accuracy: 0.9914 - val_loss: 0.0415 - val_accuracy: 0.9864
Epoch 5/5
1875/1875 [==============================] - 17s 9ms/step - loss: 0.0201 - accuracy: 0.9940 - val_loss: 0.0359 - val_accuracy: 0.9901
Example 4: Visualizing Feature Maps
Example 4: Visualizing Feature Maps
Description:
This example demonstrates how to visualize the feature maps produced by the convolutional layers, helping students understand how the model extracts different features from the input image.
# Create a model to extract feature maps
feature_map_model = models.Model(inputs=cnn_model.input, outputs=cnn_model.layers[0].output)
# Predict the feature maps for a single image
feature_maps = feature_map_model.predict(train_images[:1])
# Display the first feature map
import matplotlib.pyplot as plt
plt.imshow(feature_maps[0, :, :, 0], cmap='viridis')
plt.show()
Output
Example 5: Using Different Activation Functions
Example 5: Using Different Activation Functions
Description:
In this example, students explore the impact of using different activation functions (ReLU, Sigmoid, Tanh) in CNNs.
# Build a CNN with Sigmoid activation
cnn_model_sigmoid = models.Sequential([
layers.Conv2D(32, (3, 3), activation='sigmoid', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='sigmoid'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='sigmoid'),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
cnn_model_sigmoid.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model_sigmoid.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.6143 - accuracy: 0.8048 - val_loss: 0.1466 - val_accuracy: 0.9610
Epoch 2/5
1875/1875 [==============================] - 20s 11ms/step - loss: 0.1129 - accuracy: 0.9670 - val_loss: 0.0755 - val_accuracy: 0.9787
Epoch 3/5
1875/1875 [==============================] - 18s 10ms/step - loss: 0.0746 - accuracy: 0.9772 - val_loss: 0.0652 - val_accuracy: 0.9815
Epoch 4/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.0582 - accuracy: 0.9826 - val_loss: 0.0507 - val_accuracy: 0.9849
Epoch 5/5
1875/1875 [==============================] - 19s 10ms/step - loss: 0.0478 - accuracy: 0.9857 - val_loss: 0.0416 - val_accuracy: 0.9868
Example 6: Applying Data Augmentation in CNNs
Example 6: Applying Data Augmentation in CNNs
Description:
Students learn how to apply data augmentation techniques (e.g., rotation, flipping, shifting) to increase the diversity of training data and improve model generalization.
from tensorflow.keras.preprocessing.image import ImageDataGenerator
# Define data augmentation generator
datagen = ImageDataGenerator(
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest'
)
# Train CNN using augmented data
cnn_model.fit(datagen.flow(train_images, train_labels, batch_size=64), epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
938/938 [==============================] - 16s 17ms/step - loss: 0.9105 - accuracy: 0.6989 - val_loss: 0.1741 - val_accuracy: 0.9478
Epoch 2/5
938/938 [==============================] - 16s 17ms/step - loss: 0.4816 - accuracy: 0.8454 - val_loss: 0.1638 - val_accuracy: 0.9532
Epoch 3/5
938/938 [==============================] - 16s 17ms/step - loss: 0.3854 - accuracy: 0.8759 - val_loss: 0.1320 - val_accuracy: 0.9587
Epoch 4/5
938/938 [==============================] - 17s 18ms/step - loss: 0.3359 - accuracy: 0.8950 - val_loss: 0.1216 - val_accuracy: 0.9621
Epoch 5/5
938/938 [==============================] - 17s 18ms/step - loss: 0.3059 - accuracy: 0.9031 - val_loss: 0.1318 - val_accuracy: 0.9610
Example 7: Batch Normalization in CNNs
Example 7: Batch Normalization in CNNs
Description:
This example demonstrates how to add batch normalization layers to CNNs to speed up training and stabilize the learning process.
# Build a CNN with batch normalization
cnn_model_bn = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.BatchNormalization(),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.BatchNormalization(),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.BatchNormalization(),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
cnn_model_bn.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model_bn.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 25s 13ms/step - loss: 0.0989 - accuracy: 0.9714 - val_loss: 0.0479 - val_accuracy: 0.9841
Epoch 2/5
1875/1875 [==============================] - 35s 19ms/step - loss: 0.0410 - accuracy: 0.9873 - val_loss: 0.0375 - val_accuracy: 0.9870
Epoch 3/5
1875/1875 [==============================] - 31s 16ms/step - loss: 0.0311 - accuracy: 0.9908 - val_loss: 0.0348 - val_accuracy: 0.9886
Epoch 4/5
1875/1875 [==============================] - 30s 16ms/step - loss: 0.0240 - accuracy: 0.9922 - val_loss: 0.0286 - val_accuracy: 0.9901
Epoch 5/5
1875/1875 [==============================] - 33s 17ms/step - loss: 0.0191 - accuracy: 0.9936 - val_loss: 0.0283 - val_accuracy: 0.9905
Example 8: Exploring Different Pooling Strategies
Example 8: Exploring Different Pooling Strategies
Description:
In this example, students explore the impact of using different pooling strategies, such as max pooling and average pooling.
# Build a CNN with average pooling
cnn_model_avg_pool = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.AveragePooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.AveragePooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
cnn_model_avg_pool.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model_avg_pool.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.1770 - accuracy: 0.9469 - val_loss: 0.0598 - val_accuracy: 0.9794
Epoch 2/5
1875/1875 [==============================] - 13s 7ms/step - loss: 0.0605 - accuracy: 0.9814 - val_loss: 0.0506 - val_accuracy: 0.9834
Epoch 3/5
1875/1875 [==============================] - 13s 7ms/step - loss: 0.0426 - accuracy: 0.9870 - val_loss: 0.0331 - val_accuracy: 0.9888
Epoch 4/5
1875/1875 [==============================] - 13s 7ms/step - loss: 0.0325 - accuracy: 0.9898 - val_loss: 0.0354 - val_accuracy: 0.9892
Epoch 5/5
1875/1875 [==============================] - 13s 7ms/step - loss: 0.0263 - accuracy: 0.9917 - val_loss: 0.0335 - val_accuracy: 0.9900
Example 9: Using Global Average Pooling
Example 9: Using Global Average Pooling
Description:
This example introduces global average pooling, which reduces the spatial dimensions to a single value per feature map and is often used in modern CNN architectures.
# Build a CNN with global average pooling
cnn_model_gap = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.GlobalAveragePooling2D(),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
cnn_model_gap.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model_gap.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 7s 4ms/step - loss: 1.8879 - accuracy: 0.2962 - val_loss: 1.5794 - val_accuracy: 0.4098
Epoch 2/5
1875/1875 [==============================] - 6s 3ms/step - loss: 1.5069 - accuracy: 0.4307 - val_loss: 1.4134 - val_accuracy: 0.4759
Epoch 3/5
1875/1875 [==============================] - 7s 4ms/step - loss: 1.4305 - accuracy: 0.4638 - val_loss: 1.3951 - val_accuracy: 0.4821
Epoch 4/5
1875/1875 [==============================] - 6s 3ms/step - loss: 1.4069 - accuracy: 0.4740 - val_loss: 1.3737 - val_accuracy: 0.4804
Epoch 5/5
1875/1875 [==============================] - 7s 3ms/step - loss: 1.3897 - accuracy: 0.4796 - val_loss: 1.3424 - val_accuracy: 0.4901
Example 10: Building a Deeper CNN with More Layers
Example 10: Building a Deeper CNN with More Layers
Description:
In this final example for Week 2, students build a deeper CNN by adding more convolutional layers, experimenting with how depth affects model performance.
# Build a deeper CNN model
deep_cnn_model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(128, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
deep_cnn_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
deep_cnn_model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.1855 - accuracy: 0.9423 - val_loss: 0.0790 - val_accuracy: 0.9753
Epoch 2/5
1875/1875 [==============================] - 21s 11ms/step - loss: 0.0622 - accuracy: 0.9809 - val_loss: 0.0538 - val_accuracy: 0.9833
Epoch 3/5
1875/1875 [==============================] - 22s 12ms/step - loss: 0.0448 - accuracy: 0.9861 - val_loss: 0.0581 - val_accuracy: 0.9827
Epoch 4/5
1875/1875 [==============================] - 22s 11ms/step - loss: 0.0345 - accuracy: 0.9891 - val_loss: 0.0499 - val_accuracy: 0.9865
Epoch 5/5
1875/1875 [==============================] - 16s 8ms/step - loss: 0.0272 - accuracy: 0.9912 - val_loss: 0.0601 - val_accuracy: 0.9831
Week 2 Summary
Week 2 Summary
Objective: Dive into CNNs and understand their architecture, components, and the impact of different layers on performance.
Skills Developed:
Understanding convolution, pooling, batch normalization, data augmentation, and various activation functions.
Hands-on experience building and experimenting with different CNN architectures.
Tools: TensorFlow, Keras, Matplotlib.
These 10 examples in Week 2 offer a comprehensive introduction to CNN architecture and components, giving students the foundational skills they need to build more complex and effective deep learning models.
Page updated
Report abuse