DEEP-Hybrid-DataCloud

The DEEP-HybridDataCloud (Designing and Enabling E-Infrastructures for intensive data Processing in a Hybrid DataCloud) is a project approved in July 2017 within the EINFRA-21-2017 call of the Horizon 2020 framework program of the European Community. It will develop innovative services to support intensive computing techniques that require specialized HPC hardware, such as GPUs or low-latency interconnects, to explore very large datasets. Although the cloud model offers flexibility and scalability, it is quite complex for a scientific researcher that develops a new application to use and exploit the required services at different layers. Within the project, WP6 is going to realize the DEEP as a Service solution composed of a set building blocks (i.e. the DEEP Open Catalogue) that enable the easy development of computeintensive applications. By using this solution users will get easy access to cutting-edge computing libraries (such as deep learning and other compute-intensive techniques) adapted to leverage highend accelerators (GPUs), integrated with BigData analytics frameworks existing in other initiatives and e-Infrastructures (like the EOSC or EGI.eu). The DEEP as a Service solution will therefore lower the access barrier for scientists, fostering the adoption of advanced computing techniques, large-scale analysis and post-processing of existing data. This deliverable provides the state-of-the-art in Deep Learning (DL), Neural Network (NN) and Machine Learning (ML) frameworks and libraries to be used as building blocks in the DEEP Open Catalogue. It also presents a comprehensive knowledge background about ML and DL for largescale data mining. The deliverable states clearly the recent time-slide of ML/DL research as well as the current high dynamic development of cutting-edge DL/NN/ML software. By combining one or more of the building blocks, users will be able to describe their application requirements. The DEEP Open Catalogue will be oriented to cover divergent needs and requirements of worldwide researchers and data scientists supported by specialised hardware and the recent current-edge work on compute- and data-intensive libraries and frameworks in the era of large-scale data processing and data mining. The initial establishment towards scientific data analytic and ML/DL tools will be built based on the outcome of this document and the initial requirements from DEEP research community coming from WP2 (Deliverable D2.1). The content of the DEEP Open Catalogue will be extendable and modifiable according to user community demands.