What You Will Learn
Three things. What is actually happening inside an LLM — tokenisation, transformer architecture, and what 'parameters' really means. How to use a model via a cloud API. And how to run an open-source model on your own machine using Ollama — free, private, works completely offline.
Step 1: Tokenisation — Text Becomes Numbers
The first thing that happens when you send a message to an LLM is that your text is broken apart into tokens. A token is roughly a word or part of a word. The sentence 'The agent scheduled a meeting' does not arrive at the model as letters or words — it arrives as a sequence of number IDs.
Step 2: The Transformer — Three Stages
Every modern LLM — Claude, GPT, Llama, Gemini — uses the same underlying architecture: the transformer, introduced in 2017. The name GPT literally stands for Generative Pre-trained Transformer.
Stage 1 — Embedding
Each token ID is looked up in an embedding table and converted into a vector of hundreds of numbers. These numbers encode the meaning and context of each word.
Stage 2 — Transformer Blocks
A stack of transformer blocks processes the vectors. Each block runs Attention (which words matter most to each other?) and a feed-forward layer (pattern lookup from training). A large model may have 96 or more of these blocks stacked.
Stage 3 — Language Model Head
The final vector is mapped to a probability score for every token in the vocabulary. The model picks the most likely next token. This repeats until the answer is complete. This is why LLM responses are generated word by word.
What Does '70 Billion Parameters' Actually Mean?
Parameters are the numbers inside the model — the values learned during training. More parameters means more capacity, but also more compute required to run the model.
| Model | Size | Notes | Hosting |
|---|---|---|---|
| Llama 3.2 3B | 3B | Small but capable | Runs locally on most laptops |
| Qwen 2.5 7B | 7B | Strong general model | Runs locally on 8GB RAM |
| Llama 3.3 70B | 70B | Near-frontier quality | Needs 16GB+ VRAM locally |
| Claude Sonnet 4.6 | ~200B+ | Best quality | Cloud-only. Pay per token via API |
Two Ways to Run a Model
Online — Cloud API
You send text to a remote server. Their hardware runs the model and returns the response. Advantages: no hardware needed, access to the largest models. Disadvantages: costs money per token, your data leaves your machine.
Services: Anthropic API · OpenAI API · Hugging Face Inference API
Local — Run on Your Machine
You download the model weights to your own computer and run them using Ollama. Advantages: completely free after setup, data never leaves your machine, works offline. Disadvantages: smaller models with less capability, requires decent hardware.
📝 Your Lesson Submission
Share your reflection, ask a question, or leave a comment. Your mentor will respond.
You need to be logged in to submit your reflection.