Advanced Applied Deep Learning
Practice Course
Sheng Yun Wu
Embedded Files
Week 1: Introduction to Deep Learning and Neural Networks
Week 1: Introduction to Deep Learning and Neural Networks
In Week 1, the focus is on building a foundational understanding of deep learning and neural networks. The following 10 examples guide students through basic neural network concepts, data preprocessing, and building their first neural network model using Python and TensorFlow.
Example 1: Loading and Preprocessing Data
Description:
This example covers loading a dataset (MNIST) and preprocessing the images by reshaping and normalizing the pixel values. It is an essential step before feeding data into a neural network.
import tensorflow as tf
# Load MNIST dataset
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()
# Reshape and normalize the data
train_images = train_images.reshape((60000, 28, 28, 1)).astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1)).astype('float32') / 255
# Display shape of the dataset
print(f"Train images shape: {train_images.shape}")
print(f"Test images shape: {test_images.shape}")
Output
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11490434/11490434 [==============================] - 2s 0us/step
Train images shape: (60000, 28, 28, 1)
Test images shape: (10000, 28, 28, 1)
Example 2: Building a Simple Neural Network
Description:
This example introduces building a basic neural network using TensorFlow's Keras API. The network consists of an input layer, hidden layers with ReLU activations, and an output layer for classification.
from tensorflow.keras import layers, models
# Define a simple neural network
model = models.Sequential([
layers.Flatten(input_shape=(28, 28, 1)),
layers.Dense(128, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Display the model summary
model.summary()
Output
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_1 (Flatten) (None, 784) 0
_________________________________________________________________
dense_2 (Dense) (None, 128) 100480
_________________________________________________________________
dense_3 (Dense) (None, 10) 1290
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________
Example 3: Compiling the Model
Description:
This example explains how to compile the model by selecting an optimizer, loss function, and metrics to evaluate performance.
# Compile the model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Print the optimizer and loss
print(f"Optimizer: {model.optimizer}")
print(f"Loss function: {model.loss}")
Output
Optimizer: <tensorflow.python.keras.optimizer_v2.adam.Adam object at 0x000002213EF01C40>
Loss function: sparse_categorical_crossentropy
Example 4: Training the Neural Network
Description:
In this example, the model is trained using the training dataset. The fit() function is used to run the training loop over multiple epochs.
# Train the model
model.fit(train_images, train_labels, epochs=5, batch_size=64, validation_data=(test_images, test_labels))
Output
Epoch 1/5
938/938 [==============================] - 1s 1ms/step - loss: 0.2895 - accuracy: 0.9184 - val_loss: 0.1561 - val_accuracy: 0.9537
Epoch 2/5
938/938 [==============================] - 1s 936us/step - loss: 0.1305 - accuracy: 0.9621 - val_loss: 0.1137 - val_accuracy: 0.9663
Epoch 3/5
938/938 [==============================] - 1s 939us/step - loss: 0.0921 - accuracy: 0.9725 - val_loss: 0.0915 - val_accuracy: 0.9731
Epoch 4/5
938/938 [==============================] - 1s 947us/step - loss: 0.0703 - accuracy: 0.9793 - val_loss: 0.0801 - val_accuracy: 0.9760
Epoch 5/5
938/938 [==============================] - 1s 1ms/step - loss: 0.0557 - accuracy: 0.9836 - val_loss: 0.0743 - val_accuracy: 0.9771
Example 5: Evaluating Model Performance
Description:
This example demonstrates how to evaluate the performance of a trained model on a test dataset.
# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(test_images, test_labels)
print(f"Test accuracy: {test_acc}")
Output
313/313 [==============================] - 0s 541us/step - loss: 0.0743 - accuracy: 0.9771
Test accuracy: 0.9771000146865845
Example 6: Visualizing Training History
Description:
Here, students learn how to visualize the training and validation accuracy and loss over epochs using matplotlib, which helps in monitoring the model's performance.
import matplotlib.pyplot as plt
# Train the model and store history
history = model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
# Plot training & validation accuracy
plt.plot(history.history['accuracy'], label='train accuracy')
plt.plot(history.history['val_accuracy'], label='validation accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()
Output
Epoch 1/5
1875/1875 [==============================] - 2s 984us/step - loss: 0.0556 - accuracy: 0.9823 - val_loss: 0.0767 - val_accuracy: 0.9766
Epoch 2/5
1875/1875 [==============================] - 2s 954us/step - loss: 0.0411 - accuracy: 0.9872 - val_loss: 0.0851 - val_accuracy: 0.9751
Epoch 3/5
1875/1875 [==============================] - 2s 951us/step - loss: 0.0328 - accuracy: 0.9900 - val_loss: 0.0823 - val_accuracy: 0.9757
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0256 - accuracy: 0.9921 - val_loss: 0.0839 - val_accuracy: 0.9754
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0214 - accuracy: 0.9933 - val_loss: 0.0849 - val_accuracy: 0.9775
Example 7: Making Predictions with the Model
Description:
This example covers how to use the trained model to make predictions on new data.
# Make predictions on the test set
predictions = model.predict(test_images)
# Print prediction for the first test image
print(f"Predicted label for first test image: {predictions[0].argmax()}")
Output
Predicted label for first test image: 7
Example 8: Saving and Loading the Model
Description:
Students learn how to save the trained model to a file and reload it for future use.
# Save the model to a file
model.save('mnist_model.h5')
# Load the saved model
loaded_model = models.load_model('mnist_model.h5')
Output
No output here.
Example 9: Adding a Dropout Layer
Description:
This example introduces dropout, a regularization technique to prevent overfitting by randomly dropping neurons during training.
# Add a dropout layer to the model
model_with_dropout = models.Sequential([
layers.Flatten(input_shape=(28, 28, 1)),
layers.Dense(128, activation='relu'),
layers.Dropout(0.5),
layers.Dense(10, activation='softmax')
])
# Compile and train the model
model_with_dropout.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model_with_dropout.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 2s 807us/step - loss: 0.4057 - accuracy: 0.8792 - val_loss: 0.1761 - val_accuracy: 0.9479
Epoch 2/5
1875/1875 [==============================] - 1s 776us/step - loss: 0.2237 - accuracy: 0.9342 - val_loss: 0.1313 - val_accuracy: 0.9603
Epoch 3/5
1875/1875 [==============================] - 2s 884us/step - loss: 0.1878 - accuracy: 0.9437 - val_loss: 0.1101 - val_accuracy: 0.9668
Epoch 4/5
1875/1875 [==============================] - 2s 881us/step - loss: 0.1706 - accuracy: 0.9483 - val_loss: 0.1027 - val_accuracy: 0.9688
Epoch 5/5
1875/1875 [==============================] - 1s 790us/step - loss: 0.1509 - accuracy: 0.9535 - val_loss: 0.0940 - val_accuracy: 0.9719
Example 10: Using a Convolutional Neural Network (CNN)
Description:
In the final example of Week 1, students are introduced to the concept of convolutional neural networks (CNNs) by building a simple CNN to improve image classification performance on the MNIST dataset.
# Build a CNN model
cnn_model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(10, activation='softmax')
])
# Compile and train the CNN
cnn_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
cnn_model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))
Output
Epoch 1/5
1875/1875 [==============================] - 16s 8ms/step - loss: 0.1362 - accuracy: 0.9583 - val_loss: 0.0474 - val_accuracy: 0.9847
Epoch 2/5
1875/1875 [==============================] - 16s 9ms/step - loss: 0.0433 - accuracy: 0.9859 - val_loss: 0.0381 - val_accuracy: 0.9873
Epoch 3/5
1875/1875 [==============================] - 16s 9ms/step - loss: 0.0306 - accuracy: 0.9905 - val_loss: 0.0335 - val_accuracy: 0.9891
Epoch 4/5
1875/1875 [==============================] - 15s 8ms/step - loss: 0.0230 - accuracy: 0.9927 - val_loss: 0.0390 - val_accuracy: 0.9875
Epoch 5/5
1875/1875 [==============================] - 18s 9ms/step - loss: 0.0170 - accuracy: 0.9947 - val_loss: 0.0353 - val_accuracy: 0.9894
Week 1 Summary
Week 1 Summary
Objective: Introduce the basics of deep learning and neural networks.
Skills Developed:
Data preprocessing, building a simple neural network, understanding the training process, visualizing results, making predictions, and regularization techniques.
Tools: TensorFlow, Keras, Matplotlib.
These 10 examples provide a comprehensive foundation for students in the first week, ensuring they are well-prepared for more advanced topics in subsequent weeks.
Page updated
Report abuse