Eigen Vectors and Eigen Values Explained.

A AKSHAY
5 min readDec 6, 2020

Here in this blog I am going to explain what are eigen vectors and eigen values.

First we look into what is matrix multiplication will do.

Matrix Multiplication.

Let consider a matrix A of size (m x n) and matrix B of size (n x p). now we do a dot product of A.B will get a Matrix C of size (m x p). this is 2D (2 dimensional representation) in the same way we can thin of 3D,4D etc….

So, what does a dot product do actually, lets say we have a matrix A =[[3,1],[-1,3]] and then we apply a dot product with a vector X1= [1,0] if we take a dot product and represent this on a graph below. then the result vector is AX1 = [3,-1].

Fig 1

lets consider another vector X2 =[0,1], if we get a dot product with matrix A the the result will be AX2 = [-1,3]. which will be seen in below fig2.

So, we can think think of matrix A, which Transform a vector X1 or X2 Transformation means here matrix A rotates X1 by some angle and stretching by some factor x. This is what a matrix multiplication will do graphically.

Now we will think of a equation Ax =λx, when a dot product of vector x and A is calculated the resultant vector is a some constant time the vector x.

Now we relate our matrix multiplication with above equation which is a matrix A =[[3,1],[-1,3]] when multiplied with vector X1= [1,0] the resultant vector was AX1 = [3,-1], so this vector AX1 can’t be expressed as some constant time X1. So will find a value of vector X which satisfy the equation Ax =λx,

let vector X = [1,1] then calculate the dot product of A and X i.e A.X the resultant vector will be i.e AX = [2,2], now we can see that when dot product of A and X is some constant time X. the constant is 2. so (λ =2).

If we consider X = [1,1] only then our equation is satisfied. we can see in below equation

[[3,1],[-1,3]] . [1,1] = 2 x [1,1] = [2,2] (Ax=λx) our equation.

Here 2 is called eigen value and vector [1,1] is called eigen vector.

so, for every square matrix A there will be a vector X which when multiplied will result in a only a stretching of the original vector X by some constant value λ. so we are only interested in those values of vector x. which only stretches the original vector X. this is seen in fig 3.

Fig 3

So next question is how to find the value of vector X, so that when multiplied with matrix A results in a only stretching of vector X by some constant λ.

let see our equation — Ax =λx,

multiply identity matrix on each side and take the common factor. so our equation becomes-

(A-Iλ)x =0.

We have 2 case now one is

  1. when x =0, we are not interested in this so, next case is
  2. X ≠ 0, then determinant(A-Iλ) = 0, where (A-Iλ) is a singular matrix(Singular means determinant is zero).

lets continue with the above example only

when we substitute the values of our example with our eauation

equation 1

Which results in the below matrix form.

if we take the determinant of above matrix will get

here 2,4 are the solution to the equation (A-Iλ)x =0. where λ = 2 and λ =4 are the eigen values of vector A. so we got lambda(λ) values now how to calculate value of vector x.

by substituting the value of λ in the equation 1. we get

we got the Vector X = [1,1] in the same way we can replace value 2 with 4 and calculate the other eigen vector.

Hope after reading this all of you got the clarity about eigen values and vectors.

In the next article I will explain how Singular value decomposition(SVM) will be. and it quite interesting.

Thanks for reading…

My linkedIn A Akshay.

my other Articles.

  1. Basic of Autocovariance, Autocorrelation and Partial Autocorrelation explained.
  2. Linear Models and OLS use of cross-validation in python

Reference-

  1. AMATH 301
  2. Nallagoni Omkar(Data Scientist | ML & DL Trainer).

--

--