why are search functions on certain sites considerably worse than others? (not slower, but rather with way more inaccurate results)
In: 8
Ohhhh, if only anyone freaking knew.
I’m genuinely not sure how one could break this down in an ELI5 way, because there is a lot of stuff going on here. It’s not necessarily all very difficult to understand, it’s just very long chains of causality, and things don’t make a lot of sense if you don’t wander down that chain for quite some time.
On a technical level, searching for stuff is actually really, really difficult. Because what a computer thinks is important and what a human thinks is important are two very different issues, so you need to do a lot of translating between the two. For instance, if you searched for “I like bread”, a computer might, naively, take those words, run through the content and throw a million results at you, 99% of which are useless, because it turns out you don’t actually care about “I”, you care about “like bread”. But the computer didn’t.
That’s what some sites do that have really terrible searches, they just naively return whatever happens to match what you asked for. Which is usually extremely useless. You can do various things to improve this approach, but a lot of them are either computationally expensive, require user input or aren’t very user-friendly.
For example, people could actively tag their content, which makes it a lot easier to translate what a human wants to something a computer understands. But it requires human input and humans input lots of nonsense. Then there are algorithms that weigh content in certain ways, say, for example, they might massively devalue the “I” from the earlier example, because it’s a fairly useless data point. But, as you may imagine, that’s pretty damn tricky. So this kind of thing is just rather difficult, so a lot of sites just don’t bother with all that.
Then there are decently effective ways, like boolean searches, for example. Which allow you to construct logical connections between search terms. You know, something like:
>I AND like AND apples OR oranges
That would, effectively, search for both “I like apples” and “I like oranges” at the same time, allowing you to be vastly more precise in what you want. A big problem with that is that, even today, most people don’t find that very intuitive. It looks very simple, and it is, but there is a certain switch you have to flick in your head to think in that kind of cold logic and, presumably, that’s not the user experience people want. Frankly, I don’t know. That’s the only halfway reasonable explanation I could ever come up with for why most sites don’t even give you this basic luxury. Worse: A lot of sites do, but they don’t tell you that it actually works. Or how exactly their version works. Which isn’t awfully useful.
Then there is the big data approach, which comes in various flavors as well. Think Google. When people weren’t in a constant arms race with the algorithm, it often felt like Google could read your mind. It’s a bit more awkward these days because Google and the websites it crawls are constantly trying to mess with each other (also, there’s just waaaaay more content to go through), but the point is, you can achieve truly marvelous things this way. However, it’s also basically cutting edge, so to speak. You can’t expect a random site to rival Google.
Having said all that, the actual answer probably has more to do with the infrastructure of the internet itself. Few websites are truly built from scratch. For instance, remember forums? It’s no accident that there were like 2 designs and every forum ever looked like either one of those, same features and everything. That’s because writing the code behind something like a forum is a boatload of work, and there is little point in reinventing the wheel for every little forum. So there is a bit of a Lego principle happening here and, for unknown reasons, people just don’t seem to value searches very much. Furthermore, due to the difficulty of making a useful search in the first place, you have keep a lot of things in mind when structuring the data of your website, you can’t (usually) just take a random solution, plop it in your project and expect everything to work perfectly. For truly good results, you’ll have to fine tune a bunch.
Now, truth be told, everything I just told you is very half-true, simplified and doesn’t always apply equally to everything. As I said, it’s a really complex chain of events that affects a lot of things on the internet in general, not just searches. To tell you even more truth: I genuinely have no idea why the state of website searches is so utterly abysmal, I, too, failed to find an actually good reason for why things are like this. The only explanation I got that actually made any amount of logical sense is that, perhaps, people just generally don’t care.
Latest Answers