Eli5: what are eigen values and eigen vectors and how is it useful.
In: 55
There are good answers here already that explain how useful these concepts are and how they relate to linear algebra, physics, engineering etc. I will try to add some pure geometric intuition that hopefully fundamentally addresses why Eigenvectors are so important. **The key concept is orthogonality.**
Imagine the standard two-dimensional mathematical graph with an x and a y axis. We always draw the y axis perpendicular (at 90 degrees) to the x axis. Think about why we do this. Why not draw the axes at 45 degrees to each other, or some other angle? We could still cover the entire 2D plane with two lines at a non-zero angle and form a coordinate system, so what is special about 90 degrees? The answer is because, if the axes are perpendicular, the system has the useful property of being able to vary the x value without affecting the y value (and visa-versa). They are independent dimensions. Another word for this concept is orthogonality. **Geometrically, if two lines (or vectors) are orthogonal, then they are at 90 degrees to each other in space.**
If we think about the x and y axes as vectors, they are **x** = [1, 0] and **y** = [0, 1] respectively. Let’s put these together into a matrix: **A** = [[1,0] , [0,1]].Now look at the standard equation used to solve for the Eigen vectors and values: **A** **x** = λ **x.** This equation is asking “For the matrix **A**, what vector(s) **x** can we matrix multiply by such that it is equivalent to just multiplying with a single number λ?”
Every vector is an Eigenvector of the identity matrix, but if we have a more complicated matrix like **B** = [[a, b, …], [c, d, …]], then it’s more interesting. There may be no solutions, or there may be a finite number of solutions. If there are a finite number of solutions, then we can find those vectors, and use them to simplify calculations involving **B.**
**Eigen vectors are useful because they turn bad-guy matrix multiplication into good-guy scalar multiplication.**
If a matrix has multiple Eigenvectors, they will all be orthogonal to each other. If we have some arbitrary matrix **C** made up of N of vectors with several dimensions, many of which are not orthogonal to each other, then it would be useful if we could find a smaller number of vectors M < N that can be put together to cover the entire space more efficiently. In other words, a set of orthogonal vectors that can form a new simplified coordinate system that looks like the nice and comforting x-y graph.
If any system (mathematical, physical, economic, social, whatever) can be described with a mathematical model, and Eigenvectors exist in that model, then they can be used as fundamental building blocks that make understanding the behaviour of the system much simpler. Eigen values then tell you about the relationship between the Eigen vectors, e.g. which one is the biggest (is there dominant behaviour?), whether they have polarity (positivity or negativity), and whether they oscillate at different rates (complex eigen values).
**TL:DR: If you want simplify a nasty problem into something you can explain to a 5 year old who can do multiplication, you need Eigenvectors.**
Latest Answers