overview

Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) by introducing powerful AI systems capable of understanding and generating human-like text with remarkable fluency and coherence. These models, trained on vast amounts of data, are capable of performing a wide range of tasks: language translation, text summarizing, knowledge distillation, enabling researchers to navigate complex scientific literature more efficiently.

LLMs are only a starting point in unlocking the generative abilities of broader transformers to accelerate science: analyzing and plotting experimental data, formulating hypotheses, designing experiments, and even predicting promising research directions. To this end, modern transformers combine multi-modal data, leverage domain-specific representations, capture correlations through complex attention mechanisms (e.g., self-attention, cross-attention), compose specialized architectures (e.g., mixture of experts). They form the core of foundational models (FMs). The potential for architectural innovations has barely been tapped. 

In a quest for more emergent behavior (advanced capabilities not explicitly trained for but emerging spontaneously due to the massive scale and exposure to vast amounts of data during training), FMs' scale and complexity continuously increase, requiring larger training infrastructures and drastically escalating their energy footprint. In addition, the sharp increase in popularity of these models and the complexity of their prompting generates extreme volumes of concurrent inferences that need to be delivered at high throughput.

A consequence of this trend for science applications is that FM training and inference face two major challenges. The first challenge is the democratization of FMs. The scale, cost, and time required to train FMs and run inferences is prohibitively expensive for small and medium institutions, both in academia and industry. The second challenge is the unprecedented scale, duration, and throughput of parallel executions required for training and inferences. This new context raises many open research problems and innovation opportunities for parallel computing.   

This workshop will provide the scientific community a dedicated forum for discussing new research, development, and deployment of FMs at scale. Specifically, it aims to address high performance, scalability and energy efficiency of FMs through a combination of system-level and algorithmic aspects such as processing and curating the training data, efficient parallelization techniques (data, tensor, pipeline parallelism, multi-level memory management, redundancy elimination, etc.), effective data reduction approaches (for parameters, activation, optimization, gradients), low-overhead checkpointing and strategies to survive model spikes and other anomalies, fine-tuning and continual learning strategies, comprehensive evaluation and benchmarking, efficient batching, scheduling and caching of inference requests to serve a large number of users concurrently, strategies for prompt engineering and augmentation (e.g., RAG), applications to domain sciences.

Submission SCHEDULE

• Paper submission deadline: February 6th, 2026 AoE 

• Final notification: February 20th, 2026 AoE

• Camera-ready papers: March 6th, 2026 AoE 

SUBMISSION INSTRUCTIONS

Authors are invited to submit papers describing unpublished, original research. The workshop accepts full 8-page and short/work-in-progress 5-page papers, including references, figures, and tables. All manuscripts should be formatted using the IEEE conference-style template using a 10-point font size on 8.5x11-inch pages. All papers must be in English. We use a single-blind reviewing process, so please keep the authors' names, publications, etc., in the text. Papers will be peer-reviewed and accepted papers will be published in the IPDPS workshop proceedings.

All papers should be uploaded to the IPDPS submission portal.