Embedded Files
Finite Mixture Modeling (FMM)
Finite Mixture Modeling (FMM)
Like noise is often used to model unknown system input, one may use various hypotheses or finite mixture model to deal with the uncertain state space model or data association. Meanwhile, consensus may be sought over the cross-correlated sensors. These all drive a need for representing the probability distribution by a finite mixture of properly weighted component distributions, which arithmetically average the information gained from different models/hypotheses or from different sensors.
We are interested in the convex combination of the single-target probability density or the multiple target probability hypothesis density (PHD) from multiple sensors, namely Finite Mixture of Distributions/Densities
1. Best Fit of Mixture:
1. Best Fit of Mixture:
The fusion rule that approaches the true multi-sensor posterior by fitting the mixture of multiple sensor posteriors--or their first order moments--using various optimization distances.
see: Li T, et al. Best Fit of Mixture for Computationally Efficient Multi-sensor Poisson Multi-Bernoulli Mixture Filtering, Signal Process., Vol. 202, 2023, 108739.
Remark 1. When g(X) is the true/target distribution p(X), the above Theorem indicates that the average of the mixture fits the target distribution better than all component distributions on average. This therefore provides an information-theoretic justification for distribution mixing/averaging, whether the information diversity is due to model/association uncertainty or sensing diversity. This is regardless of the mixing/fusion weights. optimized mixing weights will accentuate the benefit of fusion.
Remark 2. When the fusing weights are properly designed, the average of the mixture may fit the target distribution better than the best component.
Since the year of 2019, worldwide independent research groups from China to Swissland, Germany, Australia, Italy, South Korea, Sweden, U.S. and Israel have reported some interesting theoretical properties and experimental advantages of the arithmetic average (AA) fusion (or the like using another name) in comparison with the prevailing geometric average (GA) fusion approach called GCI or EMD, especially in the context of RFS fusion for decentralized target tracking; see the following references in addition to those of mine at the end of this page. For example
G. Koliander, Y. El-Laham, P. M. Djuric, and F. Hlawatsch, “Fusion of probability density functions,” Proceedings of the IEEE, 110(4): 404–453, 2022
Mert Kayaalp, Yunus Inan, Emre Telatar, Ali H. Sayed, On the Arithmetic and Geometric Fusion of Beliefs for Distributed Inference , IEEE TAC @Xplore
This unexpected resonance demonstrates explicitly that the AA fusion is not only effective for RFS fusion but superior to its competitor, especially in dealing with frequent missed detection, closely distributed targets and massive sensors, and in enabling homogeneous and heterogeneous RFS fusion based on the unified PHD-AA consistency, not to mention faster/online computing. These findings based on critical theoretical studies and careful experimental studies overturn some earlier cheap influential criticizes about the AA fusion.
The above Frechet mean property of the AA and GA in the context of multitarget densities (and in RFS processes), though well known in pure math in the context of variables, is first pointed out in the following paper:
T.Li, X. Wang, Y. Liang and Q. Pan, On Arithmetic Average Fusion and Its Application for Distributed Multi-Bernoulli Multitarget Tracking, IEEE Transactions on Signal Processing, 68(1):2883-2896, 2020, IEEE eXplore
Remark 3. Since there still lacks a proper distance/divergence for the LRFS densities with different, discrete labels. As such, the above minimization can not be directly applied for the labeled RFS density distributions in general. Yet, this has been often violated in the literature.
See discussion given in: T. Li, Arithmetic Average Density Fusion - Part II: Unified Derivation For Unlabeled and Labeled RFS Fusion. IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 3, pp. 3255-3268, June 2024 @IEEE Xplore
2. Partial Consensus --- Many Could be Better than All
2. Partial Consensus --- Many Could be Better than All
T.Li, S. Sun and J.M. Corchado, Partial Consensus and Conservative Fusion of Gaussian Mixtures for Distributed PHD Fusion, IEEE Trans. Aeros. Electr. Syst., vol.55, no.5, 2150-2163, 2019. PrePrint is available @ IEEE Xplore
We propose a novel consensus notion, called "partial consensus", for distributed Gaussian mixture probability hypothesis density fusion based on a decentralized sensor network, in which only highly-weighted Gaussian components (GCs) are exchanged and fused across neighbor sensors. It is shown that this does not only gain high efficiency in both network communication and fusion computation but also significantly compensates the effects of clutter and missed detections.
The premilinary work of the above journal publication, also the earliest one of our series work in this direction, is the followng conference paper, which proposes the original idea of "combining" PHDs (and then merging or averaging) for fusing
T. Li, J.M. Corchado and S. Sun, On Generalized Covariance Intersection for Distributed PHD Filtering and a Simple but Better Alternative, FUSION’17 Xi’an, July 10-13, 2017 @ IEEE Xplore
Fig.2. Unweighted AA and GA of two weighted GMs. The dashed ellipse indicates the position of a potential target.
3. Parallel Consensus -- parallelization of filtering and communication/fusion operations
3. Parallel Consensus -- parallelization of filtering and communication/fusion operations
T. Li, F. Hlawatsch, A distributed particle-PHD filter using arithmetic-average fusion of Gaussian mixture parameters, Information Fusion，73: 111-124, 2021. Preprint arXiv:1712.06128v2 [cs.SY]
The first ever distributed filter based on parallel filtering-communication/fusion mode!
We propose a particle-based distributed PHD filter for tracking an unknown, time-varying number of targets. To reduce communication, the local PHD filters at neighboring sensors communicate Gaussian mixture (GM) parameters. In contrast to most existing distributed PHD filters, our filter employs an `arithmetic average' fusion. For particles--GM conversion, we use a method that avoids particle clustering and enables a significance-based pruning of the GM components. For GM--particles conversion, we develop an importance sampling based method that enables a parallelization of filtering and dissemination/fusion operations. The proposed distributed particle-PHD filter is able to integrate GM-based local PHD filters. Simulations demonstrate the excellent performance and small communication and computation requirements of our filter.
See also our attempt based on the GM filter:
T.Li*, M.Mallick, Q. Pan, A Parallel Filtering-Communication based Cardinality Consensus Approach for Real-time Distributed PHD Filtering, IEEE Sensors Journal, vol. 20, no. 22, pp. 13824-13832, 15 Nov.15, 2020. IEEE Xplore
Comparison between serial (a) and parallel (b) filtering-communication modes
Thanks to the importance sampling approach, the particle implementation renders a higher level of parallel filtering-communication than the GM implementation does. Further investigation is certainly of significance in the other distributed filter design, simply because the length of the green block is severely limited in practice!!
Existing serial filtering-communication/consensus protocol is just too ideal to be practical useful!
Average consensus is a good idea, but has to be adjusted to the real-time mode to be useful for online tracking.
4. More Realistic Useful Implementations on AA fusion:
4. More Realistic Useful Implementations on AA fusion:
Cardinality-consensus-based PHD filtering for distributed multi-target tracking, IEEE Signal Process. Lett. vol.26, no.1, 2019, pp.49-53. @ IEEE Xplore The communicationally cheapest solution among all distributed RFS filter
Local-Diffusion-based Distributed SMC-PHD Filtering Using Sensors with Limited Sensing Range, IEEE Sensors Journal, vol.19, no.4, 2019, pp. 1580 - 1589. @ IEEE Xplore The first paper addressed AA fusion in limited views
Distributed Bernoulli Filtering for Target Detection and Tracking Based on Arithmetic Average Fusion, IEEE Signal Processing Letters. Vol.26, no.12, pp. 1812-1816, 2019. IEEE Xplore; the backbone for developing many other AA-fusion-based Bernoulli-mixture filters such as AA-MB, AA-PMBM, AA-PMB, AA-MBM, AA-GLMB, AA-LMB and so on...
See also with different communication protocols are emploed:
Convergence of distributed flooding and its application for distributed Bayesian filtering, IEEE Trans. Signal and Information Processing over Networks, vol.3, no.3, pp. 580 - 591, 2017. @ IEEE Xplore with code link in the paper. Eq.(29) gives the first AA-fusion-based Particle fusion
A Robust Multi-Sensor PHD Filter Based on Multi-Sensor Measurement Clustering，IEEE Communications Letters, vol.22, no.10, pp. 2064-2067, 2018 @ IEEE Xplore Fusion over raw measurements.
“Distributed flooding-then-clustering: A lazy networking approach for distributed multiple target tracking,” in Proc. FUSION 2018, Jul. 2018, pp. 2415–2422. @ IEEE Xplore with code link in the paper.
5. AA Density Fusion - Part I: Statistic and information-theoretical results
5. AA Density Fusion - Part I: Statistic and information-theoretical results
The quality of estimation in the Bayesian inference manner, should be given with regard to the posterior distribution, not point estimate. Even that, the AA fusion has a close connection with with known covariance intersection, covariance union and so on.
T. Li, et al, Second Order Statistics Analysis and Comparison between Arithmetic and Geometric Average Fusion: Application to Multi-sensor Target Tracking, Information Fusion, vol. 51, pp. 233-243, 2019. @ sciencedirect
T. Li, et al, Arithmetic Average Density Fusion - Part I: Some Statistic and Information-theoretic Results. Information fusion,Volume 104, April 2024, 102199 @sciencedirect
T. Li, et al, Multi-sensor Suboptimal Fusion Student's t Filter, IEEE Transactions on Aerospace and Electronic Systems, vol. 59, no. 3, pp. 3378-3387, June 2023 . @IEEE Xplore
G. Koliander, Y. El-Laham, P. M. Djuric, and F. Hlawatsch, “Fusion of probability density functions,” Proceedings of the IEEE, 110(4): 404–453, 2022
Mert Kayaalp, Yunus Inan, Emre Telatar, Ali H. Sayed, On the Arithmetic and Geometric Fusion of Beliefs for Distributed Inference, IEEE Transactions on Automatic Control, doi: 10.1109/TAC.2023.3330405 @IEEE Xplore
The AA fusion works well with the benchmark Kalman filter and Student's t filter!
See:
T.Li*, et al, Arithmetic Average Density Fusion - Part I: Some Statistic and Information-theoretic Results. Information fusion, Volume 104, April 2024, 102199 @sciencedirect
T.Li, et al, "Multi-sensor Suboptimal Fusion Student's t Filter," in IEEE Transactions on Aerospace and Electronic Systems, vol. 59, no. 3, pp. 3378-3387, June 2023 . @IEEE Xplore
It is reasonable yet simple to approximate the KL divergence between student's t distribution by that of corresponding Gauss distributions
6. AA Density Fusion - Part II: Unified Derivation for (unlabeled and labeled) RFS Fusion
6. AA Density Fusion - Part II: Unified Derivation for (unlabeled and labeled) RFS Fusion
The arithmetic average (AA) fusion is a fundamental information fusion methodology which has recently demonstrated great performance for multi-sensor tracking of a random number of objects based on the random finite set (RFS) theory. Since there are extra multi-object set cardinality (namely the number of objects) and even identities need to be estimated jointly with the multi-object states, the AA fusion has to be tailored in various specific means to accommodate different RFS filters. All can be strictly derived from the same unlabeled/labeled probability hypothesis density (PHD) averaging formulation which ensures consistency in the (labeled) PHD estimation. In this paper, we first explain how the (labeled) PHD-consistency/AA fusion can lead to more accurate and robust detection and localization of the present objects which forms a both theoretically solid and physically meaningful reason for fusion. Then, we derive and analyze the formulas of suitable RFS-AA fusion for different (labeled) RFS filters on the same base of the (labeled) PHD-AA/consistency. These derivations are exact and need no approximation.
See:
T. Li, Arithmetic Average Density Fusion - Part II: Unified Derivation For Unlabeled and Labeled RFS Fusion. IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 3, pp. 3255-3268, June 2024 @IEEE Xplore
Realization has been shown in the following papers on Heterogeneous Fusion
The above MSE^s regards the probability volume of any local area s.
7. AA Density Fusion - Part III and IV: Heterogeneous Unlabeled and Labeled RFS Density Fusion
7. AA Density Fusion - Part III and IV: Heterogeneous Unlabeled and Labeled RFS Density Fusion
Thanks to the above fact that all AA-RFS filters are built on averaging their relevant unlabeled/labeled probability hypothesis densities (PHDs), the following paper proposes the first ever heterogenous unlabeled and labeled RFS filter cooperation approach based on Gaussian mixture implementations where the local Gaussian components (L-GCs) are so optimized that the resulting unlabeled PHDs best fit their AA, regardless of the specific type of the local densities. For illustration, a computationally efficient, approximate approach is proposed which only revises the weights of the LGCs, keeping the other parameters of L-GCs unchanged. In particular, the PHD filter, the unlabeled and labeled multi-Bernoulli (MB/LMB) filters are considered. Simulations have demonstrated the effectiveness of the proposed approach for both homogeneous and heterogenous fusion of the PHD-MBLMB filters in different configurations.
See:
T. Li, Arithmetic Average Density Fusion - Part III: Heterogeneous Unlabeled and Labeled RFS Filter Fusion, IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 1, pp. 1023-1034, Feb. 2024 @IEEE Xplore
T. Li, H. Liang, G. Li*, J. G. Herrero, Arithmetic average density fusion - Part IV: Distributed heterogeneous LRFS/RFS filter fusion via variational approximation, IEEE Trans. SP, , vol. 73, pp. 1454-1469, 2025. DOI: 10.1109/TSP.2025.3550157
Overview: Beyond AA!
From the celebrated Gaussian mixture, model averaging estimators to the cutting-edge multi-Bernoulli mixture of various forms, ...........
T.Li*, H. Liang, B. Xiao, Q. Pan and Y. He, Finite mixture modeling in time series: A survey of Bayesian filters and fusion approaches, Information Fusion, 2023, 98: 101827. URL
Google Sites
Report abuse