[TBD]Role of Mixer Model in current data science

Introduction

Laymen explanation

Technical explanation

In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observation belongs. Since subpopulation assignment is not known, this constitutes a form of unsupervised learning (clustering).

Gaussian mixer model

In real life, many datasets can be modeled by Gaussian Distribution (Univariate or Multivariate). So it is quite natural and intuitive to assume that the clusters come from different Gaussian Distributions. Or in other words, it is tried to model the dataset as a mixture of several Gaussian Distributions. This is the core idea of this model.

Use in machine learning

GMMs have been used for feature extraction from speech data, and have also been used extensively in object tracking of multiple objects, where the number of mixture components and their means predict object locations at each frame in a video sequence.

Relevance with neural networks

Both(mixer model and neural networks) are very similar to each other, for they both have hidden layers and both require iterative methods to perform inference tasks.

Neural network (in machine learning) focus mainly on minimizing the prediction error, as long as the prediction error is minimized, it doesn't matter how you interpret the mathematic equations, or how many hidden layer/nodes you used in the model. On the other hand, mixture model (in statistics) focus mainly on maximizing the marginal likelihood, and every hidden layer and node matters because each of the hidden node or layer must have a corresponding real world explanation.

Reference

https://youtu.be/ZBLyXgjBx3Q

https://en.wikipedia.org/wiki/Mixture_model

https://brilliant.org/wiki/gaussian-mixture-model/

https://www.geeksforgeeks.org/gaussian-mixture-model/

https://images.app.goo.gl/WvdDVewK9Juc1nyR6

https://stats.stackexchange.com/questions/463706/are-neural-networks-mixture-models