PaddlePaddle (PArallel Distributed Deep LEarning) is an open source, efficient, flexible and scalable DL platform, which is originally developed by Baidu scientists and engineers for the purpose of applying DL to many Baidu products [PaddlePaddle]. At Baidu, PaddlePaddle has been deployed into products and services with a vast number of users, including ad click-through rate (CTR) prediction, large-scale image classification, optical character recognition (OCR), search ranking, computer virus detection, recommendation, etc. It supports a wide range of NN architectures and optimization algorithms. It is easy to configure complex models such as neural machine translation model with attention mechanism or complex memory connection.

MatConvNet is a MatLab toolbox implementing CNNs for computer vision applications [MatConvNet] developed by the Oxford computer vision team and other research institutions. It is simple and integrating MatLab GPU support, and can run and learn CNNs with similar results to top scores in the ImageNet challenge. Many pre-trained CNNs models e.g. VGG, AlexNet for image classification, segmentation, face recognition, and text detection are available. An important feature of MatConvNet is making available the CNN building blocks as easy-to-use MatLab commands. MatConvNet does not support for RBMs and DBNs and it does not have OpenMP and OpenCL supports. The next weak point of the product is the proprietary character of MatLab.

The NVIDIA Deep Learning SDK, which is a part of the NVIDIA toolkit, provides powerful tools and libraries for designing and deploying GPU-accelerated DL applications. It includes libraries for DL primitives, inference, video analytics, linear algebra, sparse matrices, and multiGPU communications. The NVIDIA CUDA Deep Neural Network (cuDNN) library is a GPUaccelerated library of primitives for deep neural networks, It accelerates widely used DL frameworks, including Caffe2, MATLAB, CNTK, TensorFlow, Theano, PyTorch, etc. The next tools in the NVIDIA Deep Learning SDK are Deep Learning Inference Engine (TensorRT), Deep Learning for Video Analytics (DeepStream SDK), Linear Algebra (cuBLAS), Sparse Matrix Operations (cuSPARSE) and Multi-GPU Communication (NCCL).

The development of DL frameworks and libraries is quite high dynamic with many interesting involving products. The popularity/trend movement of DL frameworks and libraries at the end of 2017 are depicted in Fig. 7 borrowed from https://towardsdatascience.com. It is difficult to make forecast in this fast changing ecosystem but we can see to main trends emerging in the use of DL frameworks: 1) a trend backed by Google with uses Keras for fast prototyping and Tensorflow for production, and 2) a trend backed by Facebook which uses Pytorch for prototyping and Caffe2 for production.

The extensive number of deep learning frameworks makes it challenging to develop tools in one framework and use them in other frameworks (frame interoperability). The Open Neural Network Exchange [ONNX] tries to address this problem by introducing an open ecosystem for interchangeable AI models. ONNX is being co-developed by Microsoft, Amazon and Facebook as an open-source project and it will initially support DL frameworks Caffe2, PyTorch, MXNet and Microsoft CNTK (Fig. 8).

Return to Contemt