We often hear about algorithms that can forecast our preferences, suggest products, or even detect health issues before they become apparent. How can a machine, which doesn’t have consciousness or feelings, accurately predict something so complex as human behavior? I’m fascinated by this intersection of technology and psychology and would love to hear from anyone who can shed light on the mysterious ways these algorithms seem to ‘know’ us. What’s happening behind the scenes that makes this possible?
In: 10
Statistical models take in information about the world and use patterns in those data to make predictions. There are massive amounts of data showing that people with more education earn more money. A statistical model trying to predict your income can use that data to infer your income based on your level of education.
With a great degree of care in how the model is constructed and a massive amount of data about many different facets of the world it is possible to make very good predictions about many things. Fundamentally it’s no different from the linear regression you probably did in high school math.
There is no underlying principles nor true understanding.
What computers have is an *incredible* amount of data from incredible amounts of people and the ability to actually compare all that data to each other.
Essentially they do pattern recognition, but do it on a scale that we can scarcely comprehend.
There’s an element of brute force to it – a machine can study human behavior with an intensity and repetitiveness that humans can’t match.
And they’re not limited by logic, which is actually a very powerful attribute. If a human attempted to make these kind of connections, they would inevitably try to form logical links, like “what’s the connection between a person’s income and their buying habits?”
The computer doesn’t know or care about that. It’s just as likely to look for a connection between hair color and toothpaste preference as it is to look for one between income and purchase power. Which lets it find connections that would be completely invisible to a human researcher.
That combination – not being bound by pre-conceptions of what should or shouldn’t be connected, and being able to study truly mind-boggling amounts of data – allows them to discover things about human behavior that we would never have suspected.
It’s so powerful that people think their devices are spying on them because they’ll be thinking about buying a jeep for a week, then suddenly they start seeing jeep ads. Because it’s hard to believe that to a computer, we’re THAT predictable.
(I know there’s some spying going on too, but that’s a whole other rabbit hole, and it’s less connected to this issue than some people would probably assume. It’s much more about prediction than it is about listening to you in your home.)
There’s an economics principle that individual humans are unpredictable, but groups of humans are very predictable.
We all like to think we’re special snowflakes, but in reality there’s like 3 – 5 categories of people based on behavior, and our wants and needs are all pretty similar.
Human behavior is pretty predictable. We now have tons of data to confirm that for certain scenarios. We feed that data into a machine. The machine just makes probabilistic guesses.
You need to think in terms of patterns. If a hundred people go to a shopping centre and 66 of them take a certain route, from walking through the front door, to the coffee shop, to the music store, then to the home store, you’ve identified a pattern that 66% of the population of shoppers seem to follow. You can then utilise that pattern to do stuff that might suit your aims, such as put a certain coffee ad up on the doors, or certain music posters or certain home items. Those ads might
resonate with 66% of the people entering the mall.
That’s the principle that algorithms tap into – identifying patterns by analysing a lot of data.
I also think alot of people lie to themselves and an algorithm doesn’t take that into account.
We think we are not effected by pier pressure or commercials, we tend to underestimate the amount of alcohol, chocolate or fat we consume and we all know that we all make up our own mind about politics no matter what the news says….. Right….
An algorithm looks at the raw data and makes predictions off of that.
An add pops up, and X mount of minutes later a product has a slight rise in sales.
There’s an old saying, “Red sky at morning, sailors take warning. Red sky at night, sailors delight”.
That was a saying long before anyone understood why it worked. People just noticed a pattern after events happened many times.
But they might not notice other more complicated patterns. Red sky plus high clouds plus low sea swells plus wind from 30 degrees southwest when the ship is north of Cuba means…? Even if a single human experienced that situation more than once, that human might not remember what the results were the previous time.
But a computer remembers. And the computers we use today are given huge amounts of data, far more than any human can possibly remember. And the computer looks for patterns. When a set of events happens the computer can look through its vast amount of data to see what usually comes next.
You know how at the grocery store, they will put the hot dog buns with the hot dogs instead of with the bread? The grocer may also put ketchup, mustard, and relish there—things that you might not buy every time you buy hot dogs. The shop owner has recognized a pattern that when people buy hot dogs, they also buy hot dog buns. And he’s recognized that if people see the ketchup when picking up hot dogs, they’re more likely to buy it.
The algorithms that you’re talking about are doing the same thing as the grocer. However, they are better at pattern recognition, and they can make sense of so much more data. The artificial intelligence algorithm can analyze a ton of data to make a mapping of what customers who buy x, y, z also buy. And also customers that just looked at a, b, c buy.
Because human behaviour is not as complex as you think. Perhaps the reasoning behind the decisions are that complex but the outcome is remarkably predicable. Simply put you are not that unique
And when there’s hundreds of people making almost the same choices as you it’s not hard to guess what your next choice will be by just looking at what the people similar to you have already chosen.
Latest Answers