The 1st internationaL workshop on Near Real-time Data Processing for Interconnected Scientific Instruments

The complexity of scientific research calls for dynamic integration of various interconnected scientific instruments for data generation (e.g., experiments, observation, and simulation) and data analysis (e.g., AI/ML, visualization, etc.). The capability of near real-time data processing across interconnected scientific instruments is the foundation of various scientific workflows, including traditional human-in-the-loop and autonomous workflows. This is because analysis results are needed near real-time to provide time-sensitive decision-making and steering of experiments. However, as the improvement of scientific instruments leads to the generation of scientific data with unprecedented volumes and modalities, it imposes a huge strain on data processing as data acquisition, sharing, and analysis will be prohibitively expensive with the increase in data volumes. This landscape highlights the growing need for research efforts that optimize all stages of data processing at an extreme scale to enable near real-time processing, including but not limited to acquisition, reduction, management, storage, sharing, and analysis.

There are at least three important topics that our community is striving to answer: (1) how to design efficient data acquisition and reduction pipelines that support near-instrument preprocessing while maintaining the important features for scientific pursuit; (2) how to achieve extreme-scale data curation and sharing that leverages the advanced HPC infrastructure; (3) how to accommodate near real-time data analytics at extreme-scale with streaming or urgency requirements for time-sensitive decision making. Tackling these challenges requires expertise from computer science, mathematics, and application domains to study the problem holistically and develop solutions.

This international workshop targets HPC applications, researchers, and domain experts with big data problems and looking for new data management and analytical workflows for their applications. The outcome of this workshop will foster the implementation of near real-time data processing workflows by accelerating all stages of the scientific research lifecycle, including large-scale data acquisition, data curation, analytics, and sharing.

Workshop agenda (tentative)

Keynote speakers

Program committee (tentative)

TBD

Organization

• Jieyang Chen, University of Alabama at Birmingham

• Jinzhen Wang, Brooklyn College of City University of New York

• Qing Liu, New Jersey Institute of Technology

• Scott Klasky, Oak Ridge National Laboratory