How do they decide what to say or how to respond
In: 0
ChatGPT, one of the most famous AI “Chat bots” out there, actually works in a very similar way to how the predictive text function on your cellphone might work. It looks at all of the words given to it, and then uses its vast knowledge of word sequences to generate a list of “The most likely word to come after the one that was just typed”. It then picks one from the list, generally going with the top result. The result is something that is very human in nature.
To build up this massive index of knowledge, they essentially feed this massive algorithm part of a news article, and then ask it to finish the article. It’s then compared with the actual result, and the AI learns from its mistakes. Do this enough times, and it gets *really* good at predicting what the next word is going to be. But, you have to be careful, because if you tell the AI to do this too much, it starts just regurgitating the original text that it was fed with. So, they need to strike a middle-ground between these two where the AI can create something that’s new, but isn’t a totally random string of words.
The major difference between something like ChatGPT and the predictive text on your phone, though, is that ChatGPT has essentially virtually typed all of the data that it has seen billions and billions of times over. As a result, the predictive text algorithm is very good. It also has an advantage over traditional predictive text algorithms of being able to look back at all the words that have previously been typed in the entire conversation to determine the next response, whereas the type of algorithm used by your phone usually only looks back at a word or two, and only really learns based on *your* writing style.
Now, the dataset that ChatGPT has seen remains a secret (at least to my knowledge, someone please correct me if the dataset or the content of the dataset is available!), but it’s pretty evident that it contains a bunch of information from a ton of publicly-available forums where people go to ask questions, and get responses from actual humans like those from Microsoft, StackOverflow, and Reddit, as well as a ton of publicly-available news articles and other websites. That’s why it’s really good at generating “How-to” style responses.
Now, this isn’t all chatbots that are out there. If you visit a website that has an “Integrated chat” function that doesn’t just connect you directly to a human, odds are that it functions similar to a Google search of predefined help articles. Typically, these types of chatbots just look for specific keywords. For example, your electric company might have a chat bot that lets you enter some text, and you enter something like “I need to pay my electric bill”, and it will return an article about accessing your account and paying your bill. From your input, it may have picked out keywords such as “pay” and “bill” and decided that you want help paying your bill.
There are a ton of chatbots out there, though, and this is a very, very simplified version of how certain chatbots work. If you’re interested in learning more, please feel free to ask questions! I’ve been doing research into AI models for about 8 years now, and it’s something I’m actually very passionate about!
Latest Answers