Build a tiny LLM from scratch.
Every chatbot does one thing: guess the next piece of text. You'll build that guesser yourself, one question at a time.
You need
- ✓ Basic Python: lists, loops,
if, functions - ✓ Python and PyTorch:
pip install torch - ✓ School math: fractions, “a 50% chance”
You don't need
- ✗ Calculus
- ✗ AI experience
- ✗ A GPU
Rusty on Python? Try the Python tutorial.
Learning
- 1 Go in order.
- 2 Do each activity before reading on.
- 3 Guess before you click.
- 4 Run the Python code yourself.
Shortcuts
- G Guide me
- P Pen for notes
- ← → Back / next
- T Big text
- Esc Stop pen / guide
This is me after Lesson 5. I only read one sentence, so I repeat myself a lot.
Your model is tiny, so it learns words and grammar, not meaning. ChatGPT works the same way, just millions of times bigger. Here you learn how, line by line.
The path
Each lesson ends with a question. The next lesson answers it.