Foundations of Deep Learning for Engineering and Science, Hands on Using Python
(Learn by Coding)
Objective
Deep Learning has been a buzzword not only in the scientific community but also in netizens life!. Deep learning models have myriad applications: speech recognition , face recognition, stock market prediction and gene target in biology, just to name a few to show its diversity of applications in the scientific community . All major tech giants such as Google, Facebook, Twitter and so on have been applying deep learning models for various tasks which were once considered difficult to solve on a large scale.However, deep learning models which are popular nowadays are not new! It was there in 1980's but they are being used extensively now, only because of the availability of computational power. In a nutshell, if you have data, deep learning models might help you in whatever the problem you are dealing with.
All the variety of deep learning models such as CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) have underlying principles behind its operation. It is important to understand those principles mathematically in order to create and apply the deep learning models for a particular task at hand. The purpose of this workshop is to set the learning platform to understand those principle mathematically (Intuitively) by coding it from the scratch. We believe that once the foundation is laid, everyone can build on it on their own. Welcome to the workshop :-).
Learning Outcomes
The skill set gained after attending the workshop is called learning outcomes. We set six learning goals for the workshop. They are,
you will be able to:
Define the process of learning and how to measure it for any learning models. Understand the difference between Machine Learning and deep learning.
Preparing the data required for training and testing the model.
Understand the convex optimization and loss functions and know how learning actually takes place in the model.
Code a single neuron from the scratch and understand over fitting, under fitting and regularization.
Code DNN (Deep Neural Network) using python framework and understand the vanishing gradient problem and need to choose different activation functions.
Code CNN (Convolutional Neural Network) using python framework.
Course Schedule
Well,we have 16 Hours. We optimized the schedules to meet our learning outcomes of the workshop. Here I briefly explain what you are expected to learn in a sequential manner.
1.Introduction to learning
A brief and quick overview on the history and recent trends in deep learning era. Having said that, we define the principle of learning and show how large amount of data guarantee an exponential improvement in the performance of a learning model. In addition to that we discuss about various learning strategies in the absence of sufficient data and learn the difference between machine learning and deep learning.
2. Setting up the Coding Environment
The are plenty of programming languages available , but we choose python for its elegance. There are many ways to setup the environment with the required libraries to execute the code. And also there are many IDEs available to write the python script. All of them are fine!. We choose Anaconda python package manager for the beginners because of its simplicity in installing it (So we can save our time) and ability to create a virtual environment within the conda environment. Having installed the Anaconda, we will discuss as well as install required python packages to develop the deep learning models.
[ We preinstall everything and make the system ready for you to code directly. However, if you bring your personal system this session would help you to install those packages in your system].
3. Data set Preparation
It is often said in deep learning community that, on average , 70 % of the project time will be spend on preparing ,undertanding and pre-processing the data to make it suitable for the model. Of course, this is in fact true in our own experience as well. So, having solid understanding of the data often helps us to solve the problem at hand quickly. Since we use supervised learning models, we spend quite a some time on how to prepare data and label it properly. Subsequently, we discuss about dividing the entire data set into batches of training, validation and Testing data set.
4. Math of Learning Models
What we often see is that most coder ignore the concept of math while developing models for their problem at hand and struggle a lot latter to debug the code when they under perform. Learning mathematical component of each block in the deep learning model is as important as breath to body to keep it healthy and alive. Though it is not possible to teach every math concept in detail in 16 hours, we endeavor to give you an intuitive idea of equations involved by considering a single neuron as an example . Once if you glimpse a beauty of it, you will strive on your own to learn a lot about it over the period of time. We are sure of that!
Math concepts involved : n-dimensional space (Linear algebra), Function approximation, Optimization using back propagation.
5. Code the Math
Once if you understand the math concepts involved in learning models, code it in python to deepen your understanding. We create a single neuron to approximate a linear / non-linear functions and explore the concepts of optimization, loss and error surfaces, under fitting, over fitting and regularization. Anything in its simplest form gives us great insights. The entire deep learning process consists of only two parts : Forward Propagation and Backward Propagation. If you understand them for a single neuron, you took a giant step forward! Congratulations :-)
Forward Propagation
Back propagation using gradient descent
6. Deep Neural Networks (DNN) and Vanishing gradient problem.
Having mastered many concepts with the help of single neuron, now it is time to scale it up to have 'N' neurons. Not only that, we can create a layered network of neurons. The problems of greater complexity can be solved by such networks. Unfortunately, scaling up the network doesn't always guarantee a good performance and it also brings us a few obstacles in the learning process. One such problem is vanishing gradient problem. We discuss those problems and also show you methods to eliminate it by using various activation functions and regularization strategy.
To code DNN, of course we can start from the scratch using numpy alone, but difficulty lies in coding back propagation part . So it is wise to use some beautiful frameworks such as Tensorflow, Pytorch or Keras (still lot more is there) which takes care of back propagation part and also runs on GPU.In fact, the choice of framework doesn't matter at all once if you understand the skeleton of the model.
7. Convolutional Neural Networks (CNN)
If all the above sessions goes well as scheduled, we will take a look into Convolutional Neural Networks (CNN) which retains spatial information and also reduces the number of computation to the great extent. They are extensively used in image and video processing applications. We discuss the various CNN Architectures which made a breakthrough in solving computer vision problems once considered difficult to solve. Here is the architecture of CNN developed by us for Isolated Offline Tamil Handwritten Character Recognition .
Glimpse of the Previous Workshop
Thanks for all those who have attended and benefited out of the workshop. Based on your feedback, we try minimizing math details in the next workshop and give you hands on in python and Numpy prior to coding the DNN and CNN.