Why Do Large Language Models Hallucinate and Why Model Weights Are Not Enough?

Introduction

The lagre computer programs, like ChatGPT, Gemini, Claude and LLaMA are really good at making text that sounds like human written. Large Language Models (LLMs) can even write computer code. Large Language Models (LLMs) are very useful, for thinking, Summarize documents and answer questions.

LLM can do a lot of things. Large Language Models are not perfect. Sometimes Large Language Models make things up, this is called hallucination when Large Language Models do that.

A lot of people think that Large Language Models search the internet or a database to find the answer to a question.. That is not what Large Language Models do. Large Language Models use what they already know to come up with an answer, to a question. This is why Large Language Models sometimes get things wrong when they answer a question.

Large Language Models Predict Tokens, Not Facts

One thing that people get wrong about Large Language Models is that they know facts like humans do which is not true. Large Language Models are trained to predict what word comes next.

example:

if you ask: The capital of France is
Model response: Paris

The Large Language Model did not look up the answer in a book. It just knew that Paris is the word that comes after , The capital of France's. Large Language Models are not like fact checking machines. They are like machines that try to figure out what word comes next.

What is Hallucination?

Hallucination is when a Large Language Model makes something up.

example:

if you ask: Who won the ICC World Cup in 2038?

The Large Language Model will make something up. It will not say "I do not know". It will try to come up with an answer. The answer will not be true, this is called hallucination.

Why Hallucinations Happen?

There are a reasons why Large Language Models make things up.

1. They do not know something:- If the Large Language Model does not know something it will try to make something up.

2. The question is not clear:- If the question is not clear the Large Language Model will get it wrong.

3. The Large Language Model is out of date:- Large Language Models only know things that they learned during training. If something new happens they will not know about it.

4. They are just guessing:- Large Language Models are really good at guessing what word comes next. They are not perfect. Sometimes they get it wrong.

Knowledge is Stored Inside Model Weights

Large Language Models store what they know in something called model weights.  Model weights are like a box of numbers, When a Large Language Model is trained it learns from a lot of text. It puts what it learns into the model weights. The model weights have a lot of information in them like:

• Language patterns
• Grammar
• Facts
• How to reason
• How things are related

When you ask a Large Language Model a question it uses the model weights to come up with an answer.

Why Model Weights Are Not Enough?

Model weights are not enough because they are static.  Once a Large Language Model is trained it does not learn anything, If something new happens the Large Language Model will not know about it.  For example if a new smartphone comes out the Large Language Model will not know about it.

There are a reasons why model weights are not enough:

1. They are static:- Model weights do not change.

2. They do not have information:- Large Language Models are trained on data. They do not have access to information.

3. They have a cutoff date:- Large Language Models only know things that happened before a date.

4. Retraining is expensive:- It costs a lot of money to retrain a Large Language Model.

The Core Problem

There are two problems with Large Language Models.

1. They make things up:- Large Language Models are not perfect. They sometimes get things wrong.

2. They do not know everything:- Large Language Models only know what they learned during training. They do not know anything

This is a problem. How can a Large Language Model give you the information if it does not know anything new?

Large Language Models make things up because they are designed to predict what word comes next. They are not designed to check if something is true or not.

Another problem, with Large Language Models is that they store what they know in model weights. Model weights are static. They do not change.

This makes it hard to build AI systems that're reliable. To solve this problem people are using something called Retrieval-Augmented Generation. It helps Large Language Models to give accurate answers.