Role of Eigen vector in Current data science

Introduction

Laymen explanation

An eigenvector is a vector whose direction remains unchanged when a linear transformation is applied to it. If you are interested to know its use in machine learning, then this document helps.

Technical explanation

Matrix decompositions are a useful tool for reducing a matrix to their constituent parts in order to simplify a range of more complex operations.

Perhaps the most used type of matrix decomposition is the eigendecomposition that decomposes a matrix into eigenvectors and eigenvalues. This decomposition also plays a role in methods used in machine learning.

Properties of eigenvector

An eigenvector is a vector whose direction remains unchanged when a linear transformation is applied to it. Consider the image below in which three vectors are shown. The green square is only drawn to illustrate the linear transformation that is applied to each of these three vectors.

Note that Eigenvectors (red) do not change direction when a linear transformation (e.g. scaling) is applied to them. Other vectors (yellow) do.

is a scalar value called the ‘eigenvalue’. This means that the linear transformation on vector is completely defined by .

Uses in ML

Eigenvectors and eigenvalues have many important applications in computer vision and machine learning in general. Well known examples are

PCA (Principal Component Analysis)

It is used for feature extraction

EigenFaces

It is used for face recognition.

Role in Neural Networks

• For neural networks, eigenvector is essential for studying the dynamics of linear autoassociators and hetroassociators.

• The form of the eigenvalue distribution suggests new techniques for accelerating the learning process. The learing time of a simple neural-network model is obtained through an analytic computation of the eigenvalue spectrum for the Hessian matrix

Reference

https://www.visiondummy.com/2014/03/eigenvalues-eigenvectors/

https://machinelearningmastery.com/introduction-to-eigendecomposition-eigenvalues-and-eigenvectors/

https://images.app.goo.gl/TpR7MKSKknRhencS9

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.109.3981&rep=rep1&type=pdf

https://images.app.goo.gl/TYxNXbNi2mfUC4fb7

https://personal.utdallas.edu/~herve/Abdi-lann-01.pdf

https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.66.2396

http://proceedings.mlr.press/v97/ghorbani19b/ghorbani19b.pdf