Trang chủ‎ > ‎IT‎ > ‎Data Science - Python‎ > ‎Tensorflow‎ > ‎

Deep Learning with Keras, TensorFlow, and Anaconda

You will need to have Keras 1.2.1 and TensorFlow installed, both available as conda packages.

conda install -c conda-forge keras tensorflow

This is based on François Chollet's (author of Keras) Convolutional Neural Net example. @fchollet


                Initial Jupyter Notebook and Keras setup

In [3]:
%config InlineBackend.figure_format = 'retina'
%matplotlib inline
In [4]:
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
In [5]:
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = 10, 5

from keras.datasets import mnist
from keras.models   import Sequential
from keras.layers   import Dense, Dropout, Activation, Flatten
from keras.layers   import Convolution2D, MaxPooling2D
from keras.utils    import np_utils
from keras          import backend as K
Using TensorFlow backend.

Configure Keras to use TensorFlow and setup GPU

In [6]:
# Limit GPU memory consumption to 30%
import tensorflow as tf
from   keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.8
set_session(tf.Session(config=config))

Setup training parameters

In [7]:
batch_size         = 128
nb_classes         = 10
nb_epoch           = 10

img_rows, img_cols = 28, 28 # input image dimensions
nb_filters         = 32     # number of convolutional filters to use
pool_size          = (2, 2) # size of pooling area for max pooling
kernel_size        = (3, 3) # convolution kernel size

Load training and testing data

In [8]:
# the data, shuffled and split between train and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data()
In [9]:
print(X_train.shape)
for i in range(21):
    plt.subplot(3, 7, i+1)
    plt.imshow(X_train[i], cmap='gray_r')
    plt.title('Digit: %d' % y_train[i])
    plt.xticks([])
    plt.yticks([])
(60000, 28, 28)

Clean up the MNIST data

In [10]:
if K.image_dim_ordering() == 'th':
    X_train     = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
    X_test      = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    X_train     = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
    X_test      = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)
In [11]:
X_train = X_train.astype('float32')
X_test  = X_test.astype('float32')
X_train /= 255
X_test  /= 255
In [12]:
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
X_train shape: (60000, 1, 28, 28)
60000 train samples
10000 test samples
In [13]:
# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test  = np_utils.to_categorical(y_test, nb_classes)

Use Keras to build the neural net topology in layers

Notice all the .add() methods that construct the topology

In [14]:
tf.device('/gpu:0')  # /gpu:0 = GTX 1080, /gpu:1 = Tesla K40
model = Sequential()

model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],
                        border_mode='valid',
                        input_shape=input_shape))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adadelta',
              metrics=['accuracy'])
In [15]:
from IPython.display            import SVG
from keras.utils.visualize_util import model_to_dot
In [16]:
SVG(model_to_dot(model).create(prog='dot', format='svg'))
Warning: Could not load "/projects/demo/deeplearn/envs/default/lib/graphviz/libgvplugin_pango.so.6" - file not found

Out[16]:
convolution2d_input_1: InputLayerconvolution2d_1: Convolution2Dactivation_1: Activationconvolution2d_2: Convolution2Dactivation_2: Activationmaxpooling2d_1: MaxPooling2Ddropout_1: Dropoutflatten_1: Flattendense_1: Denseactivation_3: Activationdropout_2: Dropoutdense_2: Denseactivation_4: Activation

Use the training set to train the model over 20 epochs

In [17]:
%%time
history = model.fit(X_train, Y_train, 
                    batch_size      = batch_size, 
                    nb_epoch        = nb_epoch,
                    verbose         = 1, 
                    validation_data = (X_test, Y_test)) # validation data used to track progress
Train on 60000 samples, validate on 10000 samples
Epoch 1/10
60000/60000 [==============================] - 12s - loss: 0.4676 - acc: 0.8577 - val_loss: 0.1106 - val_acc: 0.9658
Epoch 2/10
60000/60000 [==============================] - 10s - loss: 0.1980 - acc: 0.9423 - val_loss: 0.0758 - val_acc: 0.9771
Epoch 3/10
60000/60000 [==============================] - 10s - loss: 0.1586 - acc: 0.9540 - val_loss: 0.0625 - val_acc: 0.9809
Epoch 4/10
60000/60000 [==============================] - 11s - loss: 0.1354 - acc: 0.9609 - val_loss: 0.0565 - val_acc: 0.9819
Epoch 5/10
60000/60000 [==============================] - 11s - loss: 0.1169 - acc: 0.9662 - val_loss: 0.0493 - val_acc: 0.9834
Epoch 6/10
60000/60000 [==============================] - 11s - loss: 0.1072 - acc: 0.9689 - val_loss: 0.0452 - val_acc: 0.9854
Epoch 7/10
60000/60000 [==============================] - 11s - loss: 0.0969 - acc: 0.9714 - val_loss: 0.0457 - val_acc: 0.9849
Epoch 8/10
60000/60000 [==============================] - 11s - loss: 0.0927 - acc: 0.9734 - val_loss: 0.0407 - val_acc: 0.9869
Epoch 9/10
60000/60000 [==============================] - 11s - loss: 0.0868 - acc: 0.9746 - val_loss: 0.0425 - val_acc: 0.9861
Epoch 10/10
60000/60000 [==============================] - 11s - loss: 0.0823 - acc: 0.9761 - val_loss: 0.0390 - val_acc: 0.9874
CPU times: user 2min 6s, sys: 17 s, total: 2min 23s
Wall time: 1min 52s
In [18]:
plt.plot(history.epoch, history.history['acc'])
plt.title('Accuracy');plt.xlabel('Epoch');plt.ylabel('%')
Out[18]:
<matplotlib.text.Text at 0x7ff9cc901eb8>

Now score the model using the test set

In [19]:
%%time
score   = model.evaluate(X_test, Y_test, verbose=0)
print('Test score:',    score[0])
print('Test accuracy:', score[1])
Test score: 0.0389564984303
Test accuracy: 0.9874
CPU times: user 1.45 s, sys: 192 ms, total: 1.64 s
Wall time: 937 ms

Analyze the prediction failures

In [20]:
y_predict = model.predict_classes(X_test)
fails     = y_predict != y_test
 9984/10000 [============================>.] - ETA: 0s
In [21]:
X_test_fails    = X_test[fails]
y_test_fails    = y_test[fails]
y_predict_fails = y_predict[fails]
In [22]:
for i in range(12):
    plt.subplot(3, 4, i+1)
    plt.imshow(X_test_fails[i,0], cmap='gray_r')
    plt.title('Predict: %d, Actual: %d' % (y_predict_fails[i], y_test_fails[i]))
    plt.xticks([])
    plt.yticks([])
plt.tight_layout()
Comments