Advanced Applied Deep Learning
Practice Course
Sheng Yun Wu
Practice Course
Sheng Yun Wu
In Week 1, the focus is on building a foundational understanding of deep learning and neural networks. The following 10 examples guide students through basic neural network concepts, data preprocessing, and building their first neural network model using Python and TensorFlow.
Example 1: Loading and Preprocessing Data
Description:
This example covers loading a dataset (MNIST) and preprocessing the images by reshaping and normalizing the pixel values. It is an essential step before feeding data into a neural network.
import tensorflow as tf
# Load MNIST dataset
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()
# Reshape and normalize the data
train_images = train_images.reshape((60000, 28, 28, 1)).astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1)).astype('float32') / 255
# Display shape of the dataset
print(f"Train images shape: {train_images.shape}")
print(f"Test images shape: {test_images.shape}")
Example 2: Building a Simple Neural Network
Description:
This example introduces building a basic neural network using TensorFlow's Keras API. The network consists of an input layer, hidden layers with ReLU activations, and an output layer for classification.
from tensorflow.keras import layers, models
# Define a simple neural network
model = models.Sequential([
layers.Flatten(input_shape=(28, 28, 1)),
layers.Dense(128, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Display the model summary
model.summary()
Example 3: Compiling the Model
Description:
This example explains how to compile the model by selecting an optimizer, loss function, and metrics to evaluate performance.
# Compile the model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Print the optimizer and loss
print(f"Optimizer: {model.optimizer}")
print(f"Loss function: {model.loss}")
Example 4: Training the Neural Network
Description:
In this example, the model is trained using the training dataset. The fit() function is used to run the training loop over multiple epochs.
# Train the model
model.fit(train_images, train_labels, epochs=5, batch_size=64, validation_data=(test_images, test_labels))
Example 5: Evaluating Model Performance
Description:
This example demonstrates how to evaluate the performance of a trained model on a test dataset.
# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(test_images, test_labels)
print(f"Test accuracy: {test_acc}")
Example 6: Visualizing Training History
Description:
Here, students learn how to visualize the training and validation accuracy and loss over epochs using matplotlib, which helps in monitoring the model's performance.
import matplotlib.pyplot as plt
# Train the model and store history
history = model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
# Plot training & validation accuracy
plt.plot(history.history['accuracy'], label='train accuracy')
plt.plot(history.history['val_accuracy'], label='validation accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()
Example 7: Making Predictions with the Model
Description:
This example covers how to use the trained model to make predictions on new data.
# Make predictions on the test set
predictions = model.predict(test_images)
# Print prediction for the first test image
print(f"Predicted label for first test image: {predictions[0].argmax()}")
Example 8: Saving and Loading the Model
Description:
Students learn how to save the trained model to a file and reload it for future use.
# Save the model to a file
model.save('mnist_model.h5')
# Load the saved model
loaded_model = models.load_model('mnist_model.h5')
Example 9: Adding a Dropout Layer
Description:
This example introduces dropout, a regularization technique to prevent overfitting by randomly dropping neurons during training.
# Add a dropout layer to the model
model_with_dropout = models.Sequential([
layers.Flatten(input_shape=(28, 28, 1)),
layers.Dense(128, activation='relu'),
layers.Dropout(0.5),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
model_with_dropout.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model_with_dropout.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Example 10: Using a Convolutional Neural Network (CNN)
Description:
In the final example of Week 1, students are introduced to the concept of convolutional neural networks (CNNs) by building a simple CNN to improve image classification performance on the MNIST dataset.
# Build a CNN model
cnn_model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the CNN
cnn_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Objective: Introduce the basics of deep learning and neural networks.
Skills Developed:
Data preprocessing, building a simple neural network, understanding the training process, visualizing results, making predictions, and regularization techniques.
Tools: TensorFlow, Keras, Matplotlib.
These 10 examples provide a comprehensive foundation for students in the first week, ensuring they are well-prepared for more advanced topics in subsequent weeks.