You will follow a model from raw text to a helpful assistant.
Outcomes
What You Will Be Able To Do
Explain tokens and why they matter
Describe pretraining
Explain fine-tuning and preference tuning
Explain temperature and sampling
Count tokens against a context window
Big picture
An LLM In One Paragraph
A transformer with billions of parameters
Trained to predict the next token
Produces probabilities, picks one, repeats
Everything else grows from this loop
Input
Tokens
Chunks from a fixed vocabulary
Common words are one token
Rare words split into pieces
"Unbelievably useful"
-> ["Un", "believ", "ably", " useful"]
English: about 4 characters per token
Training
Stage 1: Pretraining
Read huge collections of text and code
Predict the next token at every position
Compare with the true token, reduce loss
No human labels needed
Result
What A Base Model Is Like
Excellent at continuing text
Does not reliably follow instructions
Also absorbs errors and bias from data
prompt: "What is the capital of France?"
base: "What is the capital of Germany?
What is the capital of Spain?"
Why it works
Scale
Quality improves with size, data, and compute
Scaling-law studies found smooth trends
Chinchilla paper: grow data with model size
Bigger is not the only lever
Training
Stage 2: Fine-Tuning
Train on instruction and answer pairs
Written by people, small and curated
Teaches the shape of a helpful reply
Changes behavior more than knowledge
Training
Stage 3: Preference Tuning
People compare two answers and pick the better
A reward model learns those preferences
The LLM is trained toward higher-scored answers
This is RLHF
Caution
Side Effects Of Pleasing People
Sycophancy: agreeing too readily
Confident tone, even when wrong
Trained on rater preference, not truth
Verify claims that matter
Output
Sampling And Temperature
Greedy: always take the top token
Temperature changes the spread
Top-p keeps only the likely group
Setting
Behaviour
Good for
Low (near 0)
Picks the likeliest token, repeatable
Extraction, code, facts
Medium
Some variety
Everyday writing
High
Varied, riskier
Brainstorming, names
probs: cat 0.50 dog 0.30 fox 0.20
temperature 0.2 -> almost always cat
temperature 1.0 -> cat, dog, or fox
temperature 1.5 -> flatter, riskier
Limit
The Context Window
Maximum tokens considered at once
Includes prompt, history, documents, and answer
Text beyond the limit is dropped
No memory outside the window
Practice
Using The Window Well
Important instructions in easy-to-find spots
Middle of long inputs is used less well
Too large for the window? Retrieve pieces (week 7)
[system instructions]
[relevant documents only]
[conversation so far]
[your question] <- room left for the answer
Code
Count Tokens In Python
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
text = "Large language models read tokens, not words."
ids = enc.encode(text)
print(len(ids), ids)
print([enc.decode([i]) for i in ids])