How ChatGPT Understands your Questions?

To Understand what ChatGPT does behind the scene, first we break the word “ChatGPT” into literal meaning - because OpenAI ( company who creates ChatGPT ) does the good job naming this product - as straightforward as it would be.

So, “Chat” means - we can interact with our natural language in the form of text or voice, if you prefer talking rather than texting.”GPT” - it’s stands for Generative Pretrained Transformer, sounds crazy right but in other words it’s a transformer which can generate or predict the next text based on the pretrained text ( lots of text ), will get into each technical terms in a bit and also It’s an LLM stands for Large Language Model build by OpenAI just like Claude,Llama,Gemini,etc.

But, how LLM can understand the text, it’a a machine right, it can only be understood by machine language that is binary (0s and 1s). How it can find the meaning into it and give the response that are close to it.

That’s where Transformer model comes into picture, so, we have large text broken down into small pieces so that it can plot into 3D space and like wise token are placed near to it. Like “places” token - “Paris” or “US” are close together.

The process of breaking down text into small pieces are known as Tokenization and ploting into 3D space to get meaning out of it is known as Embeddings and we use vector calculation for that.

Most of the Tokenization done for the same text are different based on LLM or models. Although, for some character like “,” or “.” they can be common (shown above).