SuperCompCloud: 7th Workshop on Interoperability of Supercomputing and Cloud Technologies

Held in conjunction with ISC'24.

Time & Location 

16 May 2024 2:00 PM – 6:00 PM, Hamburg, Germany (Hall Y6 2nd Floor).

Please refer to the ISC'24 Schedule for the most up to date information. 

Workshop Agenda

The workshop will be held in mini-symposium format with invited speakers, shorter technical talks, and brief Q&A panels. 

• 2:00 - 2:40 PM : Nina Mujkanović (Swiss National Supercomputing Centre)

  • Invited Talk: Convergence and heterogeneity – HPC/Cloud integration and workflows

• 2:40 - 3:10 PM : Stig Telfer (StackHPC)

  • Optimising OpenStack private cloud infrastructure for numerical weather prediction workloads as part of the European Weather Cloud service at ECWMF.

• 3:10 - 3:40 PM : CJ Newburn (NVIDIA)

  • Disaggregated managed storage, layering of file over reliable block storage, secure storage – Slides

•  3:40 - 4:00 PM : Q&A Panel with first set of speakers

• 4:00 - 4:30 PM : Conference Break

• 4:30 - 5:10 PM : Christopher Edsall (University of Cambridge)

  • Invited Talk: Dawn - how we are making Cambridge's new supercomputer part of the national AI Research Resource

• 5:10 - 5:35 PM : Mark Klein (ETH Zürich / CSCS)

  • Geo-redundancy multi-site HPC infrastructure

• 5:35 - 5:50 PM : Q&A Panel with second set of speakers

• 5:50 - 6:00 PM : Workshop closing and futures - Slides

The call for participation is now closed for ISC24, but please consider submitting to our SC24 workshop or referring us to engaging and diverse speakers for the next workshop!

Invited Speakers

Nina Mujkanović (Swiss National Supercomputing Centre) 

Bio: Nina Mujkanovic is a Machine Learning Engineer at the Swiss National Supercomputing Centre (CSCS). Prior to joining CSCS, she worked as a Research Engineer in the EMEA Research Lab at first Cray and then HPE. During her studies, she specialized in advanced information processing, with a  thesis focused on creating a deep neural network for the detection of pathologies in the retina. Nina’s interests include deep learning, containerization, cloud/HPC integration, and open source.

Invited talk: Convergence and heterogeneity – HPC/Cloud integration and workflows

As compute systems become more heterogeneous, high-performance computing (HPC) and cloud system tools progressively converge. Containers, which offer an array of advantages that benefit research reproducibility and portability, have consequently seen increasing integration into supercomputing centers’ workloads. This integration remains nontrivial.

In this talk, we discuss a paradigm termed adaptive containerization, which focuses on accelerating the deployment of applications and workflows on HPC systems using containers. This entails an architecture approach in which each component chosen represents the tool best adapted to the given system and site requirements. 

We initially discuss the hardware, software, and interconnect requirements of HPC and cloud. We then delve into the HPC-specific requirements for container tools and analyze the entire containerization stack, including container engines and registries. Finally, we consider cloud orchestrator and HPC workload manager integration scenarios. The talk concludes with an outlook on the HPC/Cloud integration landscape. 

Christopher Edsall (University of Cambridge)

Bio: Christopher Edsall is the Head of Research Software Engineering at Research Computing Services at the University of Cambridge and a Principal Software Engineer in the Cambridge Open Zettascale Lab. He leads the team of research software engineers and HPC consultants with the goal of better software enabling better research. He is a co-director of the recently formed Institute of Computing for Climate Science.

Christopher studied physics at the University of Canterbury and then worked in several research institutions in New Zealand (NIWA) and the UK (National Oceanography Centre and the University of Bristol) administering thier HPC systems and upskilling researchers in software engineering to make best use of the supercomputing facilities.

Invited Talk: Dawn - how we are making Cambridge's new supercomputer part of the national AI Research Resource

Workshop Abstract and Topics

Exascale computing initiatives are expected to enable breakthroughs for multiple scientific disciplines. Increasingly these systems may utilize cloud technologies, enabling complex and distributed workflows that can improve not only scientific productivity, but accessibility of resources to a wide range of communities. Such an integrated and seamlessly orchestrated system for supercomputing and cloud technologies is indispensable for experimental facilities that have been experiencing unprecedented data growth rates. While a subset of high performance computing (HPC) services have been available within a public cloud environments, petascale and beyond data and computing capabilities are largely provisioned within HPC data centres using traditional, bare-metal provisioning services to ensure performance, scaling and cost efficiencies. At the same time, on-demand and interactive provisioning of services that are commonplace in cloud environments, remain elusive for leading supercomputing ecosystems. This workshop aims at bringing together a group of experts and practitioners from academia, national laboratories, and industry to discuss technologies, use cases and best practices in order to set a vision and direction for leveraging high performance, extreme-scale computing and on-demand cloud ecosystems. Topics of interest include tools and technologies enabling scientists for adopting scientific applications to cloud interfaces, interoperability of HPC and cloud resource management and scheduling systems, cloud and HPC storage convergence to allow a high degree of flexibility for users and community platform developers, continuous integration/deployment approaches, reproducibility of scientific workflows in distributed environment, and best practices for enabling X-as-a-Service model at scale while maintaining a range of security constraints.

This workshop will cover topics related to interoperability of supercomputing and cloud computing, networking and storage technologies that are being leveraged by use cases and research infrastructure providers with a goal to improve productivity and reproducibility of extreme-scale scientific workflows:

• Virtualization for HPC, containers technologies, and multi-tenancy 

• Storage systems for HPC and cloud technologies (on-demand and interactive)

• Resource management and scheduling systems for HPC and cloud technologies

• Software defined infrastructure for high-end computing, storage and networking

• Application environment, integration and deployment technologies

• Secure, high-speed networking for integrated HPC and cloud ecosystems

• Use cases: Extreme data and compute workflows, research infrastructure deployment

• Resiliency and reproducibility of complex and distributed workflows

• Isolation and security within shared HPC environments

• Workflow orchestration using public cloud and HPC data center resources

• Authentication, authorization and accounting for HPC and cloud ecosystems

• Workforce development for integrated HPC and cloud environments

Important Dates

• Submission deadline: April 5th

• Notification of Acceptance: March 10th

• Program Published: April 12th

Workshop Committees 

Organizing Committee

Organizing Committee (supercompcloud@googlegroups.com)

• • David Y. Hancock, Indiana University

  • Winona G. Snapp-Childs, Indiana University

  • François Tessier, Inria

  • Sadaf Alam, University of Bristol

  • Maxime Martinasso, Swiss National Supercomputing Centre

Committee members are currently being solicited for future events. If you would like to participate in the committee please contact the workshop organizers at supercompcloud@googlegroups.com.