Deep Learning December 2017

Post date: Jan 20, 2018 1:45:46 PM

1. Open source framework/platform/tools:

Keras, developed by Google engineer, is a front-end for TensorFlow, Theano, MXNet, CNTK, or deeplearning4j.

• Tensorflow - Google
• CNTK - Microsoft
• MxNet - Amazon backed; developed by CMU
• Theano - University of Montreal (to be replaced by Tensorflow)
• Torch/Pytorch, Caffe/Caffe2 - Facebook

https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software

References:

https://deeplearning4j.org/compare-dl4j-tensorflow-pytorch

https://deeplearning4j.org/neuralnet-overview

https://deeplearning4j.org/convolutionalnetwork

https://github.com/thedataincubator/data-science-blogs/blob/master/deep-learning-libraries.md

http://deeplearning.net/software/theano/

https://datahub.packtpub.com/deep-learning/top-10-deep-learning-frameworks/

https://www.cio.com/article/3193689/artificial-intelligence/which-deep-learning-network-is-best-for-you.html

2. Pr-trained models:

In practice, we use pre-trained models, then fine-tune it to fit a specific purpose.

Here is a list of popular models:

Cafe - Model-zoo

Keras List

MxNet

Tensorflow

References:

https://www.analyticsvidhya.com/blog/2017/06/transfer-learning-the-art-of-fine-tuning-a-pre-trained-model/

https://flyyufelix.github.io/2016/10/03/fine-tuning-in-keras-part1.html

https://github.com/BVLC/caffe/wiki/Model-Zoo

https://keras.io/applications/

https://github.com/dmlc/mxnet-model-gallery

https://github.com/tensorflow/models/tree/master/research/slim

https://flyyufelix.github.io/2016/10/08/fine-tuning-in-keras-part2.html

https://machinelearningmastery.com/use-pre-trained-vgg-model-classify-objects-photographs/

3. Application AI architecture

• Apps - CRM, Database and basic enterprise services
  • Scalable systems implemented in Python
• Third party service/APIs - such as Google analytic, maps, SEO; Facebook apps.
• Keras - API and Python interface
  • Python is the language of deep learning interfaces
• Backend layer
  • Tensorflow - Google
    • CNTK - Microsoft
    • MxNet - Amazon backed
    • Theano - to be replaced by Tensorflow

References:

https://en.wikipedia.org/wiki/Artificial_neural_network

4. Books, Courses and learning sites

http://www.deeplearningbook.org/

http://course.fast.ai/

http://www.fast.ai/

http://wiki.fast.ai/index.php/Main_Page

• http://wiki.fast.ai/index.php/Lesson_1_Notes

10 deep learning experts

Kaggle

• https://www.quora.com/What-are-the-best-sources-to-study-machine-learning-and-artificial-intelligence/answer/Ben-Hamner?srid=RvB7
  • https://hackernoon.com/no-kaggle-is-unsuitable-to-study-ai-ml-a-reply-to-ben-hamner-27283878cede

Deep learning in OpenCV

Some Videos:

• https://www.youtube.com/watch?v=9ziVGkt8Gg4

5. History and Theory:

http://news.mit.edu/2017/explained-neural-networks-deep-learning-0414

http://beamandrew.github.io/deeplearning/2017/02/23/deep_learning_101_part1.html

• https://towardsdatascience.com/the-10-deep-learning-methods-ai-practitioners-need-to-apply-885259f402c1
  • https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/
  • http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/
  • https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/
  • https://algobeans.com/2016/03/13/how-do-computers-recognise-handwriting-using-artificial-neural-networks/

References:

https://en.wikipedia.org/wiki/Artificial_neural_network

https://www.slideshare.net/iljakuzovkin/deep-learning-theory-history-state-of-the-art-practical-tools

https://en.wikipedia.org/wiki/Deep_learning

https://en.wikipedia.org/wiki/Convolutional_neural_network

https://machinelearningmastery.com/5-machine-learning-areas-you-should-be-cultivating/

Notes from LinkedIn:

Deep Learning & Artificial Neural Networks >> A reality-check

#DeepLearning is mostly about constructing machine learning models that work by processing "hierarchical representations of the data." This can be achieved by building Artificial #NeuralNetworks (ANNs), which are modeled from the human brain -- #neuron and #synapse structures.

Many experts believe that neural networks are overused... It seems like every developer is trying to apply #ANNs to almost everything, even #machinelearning algorithms that don’t require it.

What are some of the pros & cons of neural networks?

PROS >> Developers say:

• ANNs are relatively easy to use

• They work well for image, sound & text recognition

• They loosely mimic the brain -- this is controversial

• They're responsible for rapid development in machine learning

CONS >> Developers say:

• They're used where simpler solutions like statistical reasoning work

• They're finicky and hard to train/tune

• They work like a #blackbox and can't explain the basis for decisions

• They're not stable for real-time use with fast data & continuous learning

For complex applications -- at scale -- such as security predictions, global finance, military & weapon systems, etc., ANN technology may not yet be ready for prime-time.