It seems like they all happily make up a completely incorrect answer and never simply say “I don’t know”. It seems like hallucinated answers come when there’s not a lot of information to train them on a topic. Why can’t the model recognize the low amount of training data and generate with a confidence score to determine if they’re making stuff up?
EDIT: Many people point out rightly that the LLMs themselves can’t “understand” their own response and therefore cannot determine if their answers are made up. But I guess the question includes the fact that chat services like ChatGPT already have support services like the Moderation API that evaluate the content of your query and it’s own responses for content moderation purposes, and intervene when the content violates their terms of use. So couldn’t you have another service that evaluates the LLM response for a confidence score to make this work? Perhaps I should have said “LLM chat services” instead of just LLM, but alas, I did not.
In: Technology
When assessing the reliability of a source there are three important questions to ask:
– Who wrote it and are they an expert in this area? If you can’t identify who wrote it and if they have a relevant qualification then it is unreliable.
– Who checked it, and are they also an expert in this area? Even experts can be wrong, so having someone check the information is important, and they should also be an expert.
– How old is the information? Older information tends to be more unreliable, while newer information tends to be more reliable.
Now it is a sad fact that most information on the internet doesn’t meet even the first of these standards. Posts tend to be anonymous, and even when someone does provide a name and a claimed qualification it is nearly impossible to verify. And most posts are upvoted or downvoted by people who really have no clue what they’re on about. In reality often the correct answer is downvoted into oblivion because it is complicated and comes across as talking down to people, or simply doesn’t contain enough jokes or sarcasm.
The best source for reliable information is academic articles, but the big problem with training AI models on these is that the AI is going to end up copying their style, and it is unreadable to most people. So instead they train AIs on garbage. And as the old computing truism goes, “garbage in, garbage out” – if you train the AI on unreliable information you get unreliable answers.
The sad fact here is that what people **want** is the illusion of being informed. They want a nice simple answer to a complicated question in the minimum time. Most people don’t know the difference between a true answer and a lie…. and don’t care – just look at the TIL (today I learned) subreddit, where almost every piece of “information” I’ve read that touches on my field of expertise is wrong in important ways.
And this is why AIs don’t actually use reliable information. Because people don’t actually want it.
Latest Answers