9. April, 2022 - Virtual Conference
5th Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf 2022)
at ICPE 2022
Disclaimer: HotCloudPerf 2022 will be fully virtual. Our experience with a virtual ICPE (and HotCloudperf) 2020 and 2021 was excellent and the participants rated the experience and format very highly. For more information, please contact us at: hotcloudperf2022@easychair.org
Overview
The HotCloudPerf workshop proposes a meeting venue for academics and practitioners, from experts to trainees, in the field of cloud computing performance. The new understanding of cloud computing covers the full computational continuum from data centers to edge resources to IoT sensors and devices. The workshop aims to engage this community and to lead to the development of new methodological aspects for gaining a deeper understanding not only of cloud performance, but also of cloud operation and behavior, through diverse quantitative evaluation tools, including benchmarks, metrics, and workload generators. The workshop focuses on novel cloud properties such as elasticity, performance isolation, dependability, and other non-functional system properties, in addition to classical performance-related metrics such as response time, throughput, scalability, and efficiency.
Acknowledgement
The HotCloudPerf workshop is technically sponsored by the Standard Performance Evaluation Corporation (SPEC)’s Research Group (RG) and is organized annually by the RG Cloud Group. HotCloudPerf has emerged from the series of yearly meetings organized by the RG Cloud Group, since 2013. The RG Cloud Group group is taking a broad approach, relevant for both academia and industry, to cloud benchmarking, quantitative evaluation, and experimental analysis.
Program
All times in CEST.
14:00 Welcome
Session 1: Big Data & Microservices in the Cloud
14:15 Keynote 1: “Scaling Open Source Big Data Cloud Applications is Easy/Hard” by Paul Brebner
14:45 Floriment Klinaku, Martina Rapp, Jörg Henss and Stephan Rhode. Beauty and the beast: A case study on performance prototyping of data-intensive containerized cloud applications.
15:05 Thrivikraman V, Vishnu R Dixit, Nikhil Ram S, Vikas K Gowda, Santhosh Kumar Vasudevan and Subramaniam Kalambur. MiSeRTrace: Kernel-level Request Tracing for Microservice Visibility.
15:25 BREAK
Session 2: Optimizing Datacenters and Development
15:30 Keynote 2: “ Onefold Tuning for System and DNN Model Parameters” by Lydia Chen
16:00 Laurens Versluis and Alexandru Iosup. TaskFlow: An Energy- and Makespan-Aware Task Placement Policy for Workflow Scheduling through Delay Management.
16:20 Robert Cordingly and Wes Lloyd. FaaSET: A Jupyter notebook to streamline every facet of serverless development.
16:40 BREAK
Session 3: Serverless Computing
16:45 Keynote 3: “ Serverless Machine Learning Serving for Scalable Workflows” by Evgenia Smirni
17:15 Robert Schmitz, Danielle Lambion, Robert Cordingly, Navid Heydari and Wes Lloyd. Characterizing X86 and ARM Serverless Performance Variation: A natural language processing case study.
17:35 George Kousiouris, Chris Giannakos, Konstantinos Tserpes and Teta Stamati. Measuring Baseline Overheads in Different Orchestration Mechanisms for Large FaaS Workflows.
17:55 BREAK
18:00 Social Session with members of the SPEC RG Cloud group; open to all ICPE attendees. Bring your own drink/snacks!
18:30 END
Keynotes
Lydia Chen: Onefold Tuning for System and DNN Model Parameters
TU Delft, Netherlands
DNN learning jobs are common in today's clusters due to the advances in AI driven services such as machine translation and image recognition. The most critical phase of these jobs for model performance and learning cost is the tuning of hyperparameters, which are in turn dependent on cluster configurations. Existing tuning strategies overlook the impact of interaction of hyperparameters and systems parameters on training time and cost. In this talk, I will present two DNN tuning solutions, PipeTune and EdgeTune, which address the trade-offs between these two types of parameters by taking advantage of recurring characteristics of DNN models and pipelining the tuning of both hyper and system parameters.
Evgenia Smirni: Serverless Machine Learning Serving for Scalable Workflows
College of William & Mary, Virginia, US
Serverless computing is a new pay-per-use cloud service paradigm that automates resource scaling for stateless functions and can potentially facilitate bursty machine learning serving. Batching is critical for latency performance and cost-effectiveness of machine learning inference, but unfortunately it is not supported by existing serverless platforms due to their stateless design. Initial experiments show that without batching, machine learning serving cannot reap the benefits of serverless computing. In this talk, I will describe BATCH, a framework for supporting efficient machine learning serving on serverless platforms. BATCH uses an optimizer to provide inference tail latency guarantees and cost optimization and to enable adaptive batching support. We prototype BATCH atop of AWS Lambda and popular machine learning inference systems. The evaluation verifies the accuracy of the analytic optimizer illustrates its use for performance and cost advantages over the state-of-the-art.
Paul Brebner: Scaling Open Source Big Data Cloud Applications is Easy/Hard
Instaclustr, California, US
In the last decade, the development of modern horizontally scalable open-source Big Data technologies such as Apache Cassandra (for data storage), and Apache Kafka (for data streaming) enabled cost-effective, highly scalable, reliable, low-latency applications, and made these technologies increasingly ubiquitous. To enable reliable horizontal scalability, both Cassandra and Kafka utilize partitioning (for concurrency) and replication (for reliability and availability) across clustered servers. But building scalable applications isn’t as easy as just throwing more servers at the clusters, and unexpected speed humps are common. Consequently, you also need to understand the performance impact of partitions, replication, and clusters; monitor the correct metrics to have an end-to-end view of applications and clusters; conduct careful benchmarking, and scale and tune iteratively to take into account performance insights and optimizations. In this presentation, I will explore some of the performance goals, challenges, solutions, and results I discovered over the last 5 years building multiple realistic demonstration applications. The examples will include trade-offs with elastic Cassandra auto-scaling, scaling a Cassandra and Kafka anomaly detection application to 19 Billion checks per day, and building low-latency streaming data pipelines using Kafka Connect for multiple heterogeneous source and sink systems.
Topics
Methodological and practical aspects of software engineering, performance engineering, and computer systems related to hot topics in cloud performance.
Empirical performance studies in cloud computing environments and systems, including observation, measurement, and surveys.
Performance analysis using modeling, simulation, and queueing theory for cloud environments, applications, and systems.
Tuning and auto-tuning of systems operating in cloud environments, e.g., auto-tiering of data or optimized resource deployment.
Software patterns and architectures for engineering cloud performance, e.g., serverless.
End-to-end performance engineering for pipelines and workflows in cloud environments, or of applications with non-trivial SLAs.
Tools for monitoring and studying cloud computing performance.
General and specific methods and methodologies for understanding and engineering cloud performance.
Serverless computing platforms and microservices in cloud datacenters.
Case studies on cloud performance and its interaction with the computational continuum
Important Dates
January 15 January 27, 2022
January 20 January 27, 2022
February 25, 2022
May 4, 2022
April 9, 2022
Abstract due
Papers due
Author Notification
Camera-ready deadline
Workshop day
Submission Types
Full-papers (8 pages including references)
Short-papers (4 pages including references)
Talk only (1-2 pages, not included in the proceedings).
Format
The format of the submissions is single-blind and should follow the ACM format of the companion conference, ICPE.
All presented papers will have a good amount of time allocated for Q&A plus feedback. In addition, the presentation session will be wrapped up by a 10-15 min discussion.
Submission Site
Articles and talk-only contributions are required to be submitted via EasyChair.
Call for Papers
You can find the full Call for Papers (CfP) here: CfP
Organizing Committee
Cristina L. Abad, Escuela Superior Politécnica del Litoral, Ecuador, (cabadr@espol.edu.ec)
Simon Eismann, University of Würzburg, Germany, (simon.eismann@uni-wuerzburg.de)
Alexandru Iosup, VU Amsterdam, the Netherlands (a.iosup@vu.nl)
Program Committee
Cristina Abad, Escuela Superior Politecnica del Litoral
Ahmed Ali-Eldin, Chalmers | University of Gothenburg
Marta Beltran, Universidad Rey Juan Carlos
Andre Bondi, Software Performance and Scalability Consulting LLC
Marc Brooker, Amazon Web Services
Lucy Cherkasova, ARM Research
Wilhelm Hasselbring, University of Kiel
Nikolas Herbst, University of Würzburg
Riccardo Pinciroli, Gran Sasso Science Institute
Alexandru Iosup, Vrije Universiteit Amsterdam
Alessandro Papadopoulos, Mälardalen University
Joel Scheuner, Chalmers | University of Gothenburg
Petr Tůma, Charles University
Alexandru Uta, Leiden University
Erwin van Eyk, Vrije Universiteit Amsterdam
André van Hoorn, University of Hamburg
Chen Wang, IBM