We often hear about algorithms that can forecast our preferences, suggest products, or even detect health issues before they become apparent. How can a machine, which doesn’t have consciousness or feelings, accurately predict something so complex as human behavior? I’m fascinated by this intersection of technology and psychology and would love to hear from anyone who can shed light on the mysterious ways these algorithms seem to ‘know’ us. What’s happening behind the scenes that makes this possible?
In: 10
There’s an element of brute force to it – a machine can study human behavior with an intensity and repetitiveness that humans can’t match.
And they’re not limited by logic, which is actually a very powerful attribute. If a human attempted to make these kind of connections, they would inevitably try to form logical links, like “what’s the connection between a person’s income and their buying habits?”
The computer doesn’t know or care about that. It’s just as likely to look for a connection between hair color and toothpaste preference as it is to look for one between income and purchase power. Which lets it find connections that would be completely invisible to a human researcher.
That combination – not being bound by pre-conceptions of what should or shouldn’t be connected, and being able to study truly mind-boggling amounts of data – allows them to discover things about human behavior that we would never have suspected.
It’s so powerful that people think their devices are spying on them because they’ll be thinking about buying a jeep for a week, then suddenly they start seeing jeep ads. Because it’s hard to believe that to a computer, we’re THAT predictable.
(I know there’s some spying going on too, but that’s a whole other rabbit hole, and it’s less connected to this issue than some people would probably assume. It’s much more about prediction than it is about listening to you in your home.)
There is no underlying principles nor true understanding.
What computers have is an *incredible* amount of data from incredible amounts of people and the ability to actually compare all that data to each other.
Essentially they do pattern recognition, but do it on a scale that we can scarcely comprehend.
There’s an economics principle that individual humans are unpredictable, but groups of humans are very predictable.
We all like to think we’re special snowflakes, but in reality there’s like 3 – 5 categories of people based on behavior, and our wants and needs are all pretty similar.
Human behavior is pretty predictable. We now have tons of data to confirm that for certain scenarios. We feed that data into a machine. The machine just makes probabilistic guesses.
Statistical models take in information about the world and use patterns in those data to make predictions. There are massive amounts of data showing that people with more education earn more money. A statistical model trying to predict your income can use that data to infer your income based on your level of education.
With a great degree of care in how the model is constructed and a massive amount of data about many different facets of the world it is possible to make very good predictions about many things. Fundamentally it’s no different from the linear regression you probably did in high school math.