Achievement

We propose a score-based generative sampling method for solving the nonlinear filtering problem with superior accuracy. A major drawback of existing nonlinear filtering methods, e.g., particle filters, is the low accuracy in handling high-dimensional nonlinear problems. To overcome this issue, we incorporate the score-based diffusion model into the recursive Bayesian filter framework to develop a novel score-based filter (SF). The key idea of SF is to store the information of the recursively updated filtering density function in the score function, instead of storing the information in a set of finite Monte Carlo samples (used in particle filters and ensemble Kalman filters). By leveraging the reverse-time diffusion process, SF can generate unlimited samples to characterize the filtering density. An essential aspect of SF is its analytical update step, gradually incorporating data information into the score function. This step is crucial in mitigating the degeneracy issue faced when dealing with very high-dimensional nonlinear filtering problems. Three benchmark problems are used to demonstrate the performance of our method. In particular, SF provides surprisingly impressive performance in reliably capturing/tracking the 100-dimensional stochastic Lorenz system that is a well-known challenging problem for existing filtering methods.

Publications:

• H. Rafid, J. Yin, Y. Geng, S. Liang, F. Bao, L. Ju, G. Zhang, A Scalable Training-Free Diffusion Model for Uncertainty Quantification, Proceedings of the SC '24 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, 2024. [Download, DOI:10.1109/SCW63240.2024.00057]

• J. Yin, S. Liang, S. Liu, F. Bao, H. Chipilski, D. Lu, G. Zhang, A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics, Proceedings of the SC '24 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, 2024. [Download, DOI:10.1109/SCW63240.2024.00009]

• S. Liang, H. Tran, F. Bao, H. Chipilski, P.J. van Leeuwen, G. Zhang, Ensemble score filter with image inpainting for data assimilation in tracking surface quasi-geostrophic dynamics with partial observations, submitted (https://arxiv.org/abs/2501.12419). 

• Z. Xiong, S. Liang, F. Bao, G. Zhang, H. Chipilski, On the sensitivity of different ensemble filters to the type of assimilated observation networks, submitted (https://arxiv.org/abs/2505.04541).

• F. Bao, H. Chipilski, S. Liang, G. Zhang, J. Whitaker, Nonlinear ensemble filtering with diffusion models: application to the surface quasi-geostrophic dynamics, Monthly Weather Review, 153(7), pp. 1155–1169, 2025. (DOI: 10.1175/MWR-D-24-0069.1)

• F. Bao, Z. Zhang, G. Zhang, A unified filter method for jointly estimating state and parameters of stochastic dynamical systems via the ensemble score filter, Communications in Computational Physics, accepted, 2024. 

• F. Bao, Z. Zhang, G. Zhang, A score-based filter for nonlinear data assimilation, Journal of Computational Physics, 514, pp. 113207, 2024.

• F. Bao, Z. Zhang, G. Zhang, An ensemble score filter for tracking high-dimensional nonlinear dynamical system, Computer Methods in Applied Mechanics and Engineering, 432, Part B, 117447, 2024.