Research

Deep emulators for scientific simulations (2022)

This work proposes a deep learning-based emulator for the efficient computation of the coupled viscous Burgers' equation with random initial conditions. In a departure from traditional data-driven deep learning approaches, the proposed emulator does not require a classical numerical solver to collect training data. Instead, it makes direct use of the problem's physics. Specifically, the model emulates a second-order finite difference solver, i.e., the Crank–Nicolson scheme in learning dynamics. A systematic case study is conducted to examine the model's prediction performance, generalization ability, and computational efficiency. The computed results are graphically represented and compared to those of state-of-the-art numerical solvers.

A Bayesian Deep Learning Approach to Near-Term Climate Prediction (2022)

Since model bias and associated initialization shock are serious shortcomings that reduce the prediction skill in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting the natural variability of the North Atlantic sea surface temperature (SST) on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model (CESM2). While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of ML models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

Visual Analytics for Performance Anomaly Detection (2019-2023)

Due to the limits I/O systems currently impose on high-performance computing systems, a new generation of workflows that include online data reduction and analysis is emerging. To diagnose their performance requires sophisticated performance analysis capabilities, due to the complexity of execution patterns and underlying hardware.

To date, no tool could handle the voluminous performance trace data needed to detect potential problems. This work introduces Chimbuko, a performance analysis framework that provides real-time, distributed, in-situ anomaly detection. As a part of this effort, we refined our visualization module to support the new scale requirement. A detail documentation can be found here.

• Part of ECP CODAR Project

Visual Analytics for Performance Anomaly Detection (2018-2019)

Anomalous runtime behavior detection is one of the most important tasks for performance diagnosis in High Performance Computing (HPC). Most of the existing methods find anomalous executions based on the properties of individual functions, such as execution time. We improve upon the existing anomaly detection approaches by utilizing the call stack structures of the executions, which record rich temporal and contextual information. With our call stack tree (CSTree) representation of the executions, we formulate the anomaly detection problem as finding anomalous tree structures in a call stack forest. Structural and temporal visualizations of CSTrees are provided to support users in the identification and verification of the anomalies during an active anomaly detection process.

• Part of ECP CODAR Project

Performance Visualization for Scientific Workflows (2017)

Visualization and especially visual analytics are useful and inevitable techniques for the exascale computing to enable a human-centered experience. In this work, we present an offline visual analytics framework for performance evaluation of scientific workflows. We first incorporate TAU (Tuning and Analysis Utilities) instrumentation tool and improve it to accommodate workflow measurements. Then we establish a web-based visualization framework, whose back end handles data storage, query and aggregation, while front end presents the visualization and takes user interaction. In order to support the scalability, a few level-of-detail mechanisms are developed. Finally, a chemistry workflow use case NWChem is adopted to verify our methods. 

• Part of ECP CODAR Project

MultiSciView: Multivariate X-ray Scattering Image Exploration (2018-2019)

We develop MultiSciView, a multivariate scientific x-ray image visualization and exploration system for beamline-generated x-ray scattering data. Our system is composed of three complementary and coordinated interactive visualizations to enable a coordinated exploration across the images and their associated attribute and feature spaces. The first visualization features a multi-level scatterplot visualization dedicated for image exploration at attribute, image, and pixel scales. The second visualization is a histogram-based attribute cross filter by which users can extract desired subset patterns from the data. The third one is an attribute projection visualization designed for capturing material global attribute correlations.

• Collaborated with Dr. Kevin Yager from CFN/BNL and Dr. Klaus Mueller from SBU.

ColormapND: Data-driven Color Mapping for Scientific Images Integration (2017)

We develop a colorization tool that automatically assigns colors to data points where similar chemical ratios get similar colors. Users can interactively correspond the values of the same data point in separate variables, manipulate how data points are distributed in the colorspace by altering color assignments based on domain knowledge, and enhance the contrast to show more details to differentiate even subtle ratio changes. This tool provides the scientists a convenient way to integrate multivariate 2D/3D sample data into one pseudo-color display, and analyze the sample component in an interactive way. This is extremely useful when the dimension of data is high.

• Collaborated with Dr. Hanfei Yan, Dr. Xiaojing Huang from NSLS-II/BNL, Dr. Klaus Mueller from SBU.

Feature Importance Methods for Climate Emulator Explanation (2021-present)

We present a study using a class of post-hoc local explanation methods i.e., feature importance methods for "understanding'' a deep learning emulator of climate. Specifically, we consider a multiple-input-single-output emulator that uses a DenseNet encoder-decoder architecture and is trained to predict interannual variations of sea surface temperature (SST) at 1, 6, and 9 month lead times using the preceding 36 months of (appropriately filtered) SST data. First, feature importance methods are employed for individual predictions to spatiotemporally identify input features that are important for model prediction at chosen geographical regions and chosen prediction lead times. In a second step, we also examine the behavior of feature importance in a generalized sense by considering an aggregation of the importance heatmaps over training samples. We find that: 1) the climate emulator's prediction at any geographical location depends dominantly on a small neighborhood around it; 2) the longer the prediction lead time, the further back the "importance'' extends; and 3) to leading order, the temporal decay of "importance'' is independent of geographical location. An ablation experiment is adopted to verify the findings. From the perspective of climate dynamics, these findings suggest a dominant role for local processes and a negligible role for remote teleconnections at the spatial and temporal scales we consider. From the perspective of network architecture, the spatiotemporal relations between the inputs and outputs we find suggest potential model refinements.

• Collaborated with Dr. Shinjae Yoo from BNL and Dr. Balu Nadiga from LANL.

A VisLRP Designer for Explainable Machine Learning (2020-2021)

Layer-wise Relevance Propagation (LRP) methods are widely used in the explanation of deep neural networks (DNN), especially in computer vision field for interpreting the prediction results of convolutional neural networks (CNN). Multiple LRP variations utilize a set of relevance backpropagation rules with various parameters. Moreover, composite LRPs apply different rules on segments of CNN layers. These features impose great challenge for users to design, explore, and find suitable LRP models. In this work, we develop a visual model designer, named as VisLRP, which helps LRP designers and students efficiently perform these tasks. 

• Collaborated with Dr. Ye Zhao from KSU.

LRP for GAN Interpretation (2020)

This preliminary research focuses on understanding the behaviors of one of the network's major components, the Discriminator, which plays a vital role but is often overlooked. Specifically, we propose an enhanced Layer-wise Relevance Propagation (LRP) algorithm called Polarized-LRP. It generates a heatmap-based visualization highlighting the area in the input image that contributes to the network decision. It consists of two parts i.e. a positive contribution heatmap for the images classified as ground truth and a negative contribution heatmap for the ones classified as generated. As a use case, we have chosen the field of astronomy, specifically the deblending of two overlapping galaxy images via a branched GAN model. Using the Galaxy Zoo dataset we demonstrate that our method clearly reveals the attention areas of the Discriminator to differentiate generated galaxy images from ground truth images, and outperforms the original LRP method. To connect the Discriminator's impact on the Generator, we also visualize the attention shift of the Generator across the training process. 

• Collaborated with Dr. Yuewei Lin from CSI/BNL and Dr. Klaus Mueller from SBU.

Visual Understanding of Multiple Attributes Learning Model (2020)

This work presents a visualization system, which is designed for domain scientists to visually understand their deep learning model of extracting multiple attributes in x-ray scattering images. The system focuses on studying the model behaviors related to multiple structural attributes. It allows users to explore the images in the feature space, the classification output of different attributes, with respect to the actual attributes labelled by domain scientists. Abundant interactions allow users to flexibly select instance images, their clusters, and compare them visually in details. 

• Collaborated with Dr. Ye Zhao from KSU, Dr. Kevin Yager from CFN/BNL.

Cross-modal Scientific Image Matching using Deep Learning (2019)

The different experimental types and measurement modalities yield different kinds of images, which provide distinct information about the structure and properties of the probed material. The collection of these x-ray images forms a unique heterogeneous set revealing material characteristics such as elemental, chemical or physical ones, across multiple length scales. These data, when combined, can provide a comprehensive view of the sample under study. In this project, we focus on the cross-modal image matching and registration using deep learning.

• Collaborated with Dr. Hanfei Yan, Dr. Xiaojing Huang from NSLS-II/BNL, Dr. Yuewei Lin from CSI/BNL

Evolutionary Visual Analysis for Deep Neural Networks (2017)

We develop a scalable level-of-detail visualization system for deep learning visualization. Our framework contributes to the visualization of training process of the models. It is built upon two newly defined quantitative metrics, discriminability and density defined to enable an insightful layer and neuron evaluation. Based on these metrics, we uncover the evolution of deep neural networks on three granularities or levels of scale – network (macroscopic), layer (mesoscopic) and neuron (microscopic). Thus a multifaceted visual analysis is provided through out the training details of the networks.

• Collaborated with Dr. Klaus Mueller from SBU.

Extending 2D/3D visual analysis into immersive environment (2018)

We use HTC Vive headset to create a virtual scientific data analysis environment and interaction. Visual analytics glyphs of the heterogeneous scientific datasets are rendered to the 3D virtual world and put scientists in the center of control to steer and interact.

• Collaborated with Dr. Bo Sun from Rowan University.

Low-dose CT Image Restoration with Reference Image Database

(i) Proposed a non-local means based de-noising method employing artifact-free and artifact-matched reference image pairs as priors to overcome the noise and streak artifacts in low-dose CT scans; (ii) devised and implemented image database query framework including SIFT-flow registration, dense SIFT feature generation, kd-tree, k-means clustering, vector quantization and histogram intersection; (iii) extended the work to patch database matching.

GPU-accelerated Regularization and Parameter Tuning for Iterative CT Reconstruction

(i) Compared a few nonlinear neighborhood filters – bilateral, trilateral, non-local means and adaptive non-local means in the use of reconstruction process in CT. They were applied as non-iterative regularization step instead of traditional total variation minimization. (ii) Devised two parameter learning strategies: exhaustive benchmark test emphasizing either time or quality and multi-objective optimization served for various tradeoff options. A new perceptual image quality metric was also presented.

Iterative Ordered Subset Reconstruction Algorithms and its Application on GPUs

(i) Generalized two algebraic CT reconstruction algorithms into an ordered subset scheme and discovered extra constraints on its real-time performance due to the mapping to the architecture and programming model of GPUs. (ii) Devised a specific framework for a low-dose reconstruction case - Electron Tomography with long object compensation and some performance speedup techniques.