Is this true?: “If the distance between two items is high but it is in the direction of low variance then they are not so dissimilar? While on the other hand if distance between those two items is high and it is in the direction of high variance then they are actually dissimilar?”

Question

I don’t know if I understood correctly from the professor.

I am studying big data, and professor was talking about similarity between different items of the same dataset. Image he was talking about: https://ibb.co/5RKFQLX

In: 0

Answer 1

Yes I think so. I was able to recognize what you were asking right away and I provided a specific example of how that concept is used with spatial data in my field. We also use PCA extensively in image analysis to look for anaomalies and reduce noise. Since PCA projected data is projected along the line of the data variance, a point that lies at the extreme far end is less similar to the mean point value compared to a point on the low end of the axis. In this case, we are looking at spectral variance rather than spatial variance in finding anomalous materials 😉