What are eigenvalues and eigenvectors, and how are they used in Principal Component Analysis?
In: Mathematics
PCA is a way of simplifying information without losing too much.
Let’s say you’re trying to buy a house. Each property has a ton of factors at play – a price, a location, a size, number of bathrooms, etc.
How should you think about all of these numbers? Well maybe a lot of them are just capturing a few underlying things. The cost and size and bathrooms are all basically about value. Maybe you can get a ‘value score’ for each house by adding up 1% of the price, the number of bathrooms and 10% of the square footage.
Now instead of three numbers, you just have one combined value metric. This is what PCA does. It helps you find the most interesting ‘scores’ you can give that will capture the ways that your properties vary.
An eigenvector in PCA is just the way a score is calculated. In my example it would be .01*price, 1*baths, .1*size. The eigenvalue is the score itself. Maybe a house gets a 1600 – that’s the eigenvalue.
If you’re a visual thinker this all shakes out nicely in spatial terms. Imagine plotting price, size and baths in 3D. If they’re very correlated, most of your properties will lie along a rough line diagonal to all of those axes (they all go up and down together). The diagonal axis they’re spread along is the eigenvector. PCA just rotates your axes so that you can read the distance along this compound direction more easily.
Latest Answers