Deep Learning reading list: http://deeplearning.net/reading-list/
Alex Krizhevsky Homepage: http://www.cs.toronto.edu/~kriz/
From http://benanne.github.io/2014/04/03/faster-convolutions-in-theano.html
Quite a few libraries offering GPU-accelerated convnet training have sprung up in recent years: cuda-convnet, Caffe and Torch7 are just a few. I’ve been using Theano for anything deep learning-related, because it offers a few advantages:
The combination of these two advantages makes Theano ideal for rapid prototyping of machine learning models trained with gradient descent. For example, if you want to try a different objective function, just change that one line of code where you define it, and Theano takes care of the rest. No need to recompute the gradients, and no tedious optimisation to get it to run fast enough. This is a huge time saver.
OverFeat: http://cilvr.nyu.edu/doku.php?id=software:overfeat:start
*Caffe*:
Writing a fully connected layer is also simple:
layers { name: "ip1" type: INNER_PRODUCT blobs_lr: 1. blobs_lr: 2. inner_product_param { num_output: 500 weight_filler { type: "xavier" } bias_filler { type: "constant" } } bottom: "pool2" top: "ip1" }
This defines a fully connected layer (for some legacy reason, Caffe calls it an innerproduct
layer) with 500 outputs. All other lines look familiar, right?