When you are first exposed to ChatGPT, you may think you are talking to a knowledgeable entity.
Even if you argue that it can’t pass the Turning test (yet), the compelling evidence is that the AI or bot at the other end of ChatGPT understands what you are saying.
Somehow it manages to retrieve knowledge from its vast database of information. It even seems to have a great imagination if you ask it to create a new poem or song lyrics from scratch.
But the reality is that the behavior you observe is nothing more than regurgitated words and blocks of text – based on rules and patterns with no actual intelligence behind them.
It’s as far as you can get from achieving a sentient AI or Artificial General Intelligence (AGI).
And, if there is no intelligence, does it mean we are heading down a path to nowhere?
What is ChatGPT and a Large Language Model (LLM)?
Based on a complex algorithm, Large Language Models (or GPT) use the theory of probability and statistical analysis to generate sentences based on the text that appears before it.
When you ask ChatGPT a question, it gathers the text you have presented and looks for words that would follow in a normal situation.
In the case of a question, it would be a suitable answer. In the case of prose or writing, it may be the continuation of a paragraph.
It can come up with coherent (and more often than not ) accurate answers or text because of the sheer volume of examples it was trained on.
When you use an algorithm on 1 or two billion pieces of information, the computing power is capable of noticing patterns that may be unobservable to the human eye.
But if you think about how your brain processes a sentence, the algorithm works similarly.
An example of thought
If I start a sentence with “John has a “, your brain will automatically think of things John may have in his possession.
The things you come up with will be based on your view of the world or the knowledge you have built up over the years.
If we limit the choices to animals, you may think of the words cat, dog, bird, etc. You may also think of other less common animals, including snakes, elephants, tigers, and a sloth (from the amazon jungle).
Based on your knowledge, you will probably think that it is less likely that he has a sloth than the other options. You may think that more people have dogs than cats (or visa-versa)
Based on your knowledge and making an educated guess, you may say that “John has a dog.”
You have used statistics to make an “educated” guess.
A slight change can modify the answer
Now, if I changed the original text slightly and said that “John has a yellow …”, your logic will change.
You may come up with the same answers, but the order of the options will differ. The chances of the bird being yellow are now higher than having a yellow dog.
If you live in the desert, you may feel that a yellow snake is more likely than seeing a bird so far from the sea.
So now you may make the educated guess that “John has a yellow snake.”
This is a straightforward example. But by analyzing the text at the start of the query, we can use the process of elimination (based on probability) to eliminate ideas and rank the ones we have left.
It does just apply to objects. It can apply to any word within a sentence.
Fine Tuning GPT and LLM
If I ask the question, “Which city is the Eiffel Tower located in?” you don’t need the entire sentence to figure out the answer.
All you need are the words “which” and “Eiffel” to make an educated guess of the answer.
If we convert each word into a token, these two words or tokens are the most important part of the sentence.
If we consider 1.75 billion examples of text, the chances are incredibly high that there are hundreds or thousands of examples where these two words are followed by the word “Paris”.
However, some samples may have a spelling mistake, or the word may be lowercase.
Paris – 45% of the time
paris – 7%
Parris – 0.1%
(These stats are not real)
The rules may have also noticed that these words are followed with
France – 2%
Europe – 8%
And so on…
By using statistics, the AI can use the overwhelming probability that when it sees the words “Which” and “Eiffel” together, you are asking a question that needs to be answered with the word “Paris.”
There was no real intelligence involved. It was all mathematics!
If we don’t want the answer to be Paris, we can train the Large Language Model by adding extra text into the mix. The fine-tuning rules can be given a bias to ensure that they take priority over the other 1.75 billion examples the model already has.
Is it worth pursuing GPT and LLMs?
GPT is the first time the general public has become aware of the remarkable results you can get from computation pattern matching.
Considering that it is based on mathematics and statistics, it is amazing it works at all.
However, it is a long way from the sentient AI everyone is worried about.
LLM is one avenue of AI. They are excellent for conversational bots and creative writing. A well-trained model can answer questions and use patterns to develop outlines and business documents.
It’s essential that the technology continue to develop further and become fine-tuned.
LLM’s ability to help in education, health, and business will continue to be game-changing events.
Meanwhile, scientists can continue working on Artificial General Intelligence (with logic trees and neural networks) to develop solutions that can offer reasoning, common sense, planning, and self-learning.