Let the model learn

The problem

Counting got me stuck. Is there another way to find my numbers?

Right now your model is stuck:

• It sees only the last letter, so it writes words like “sathe”.
• Looking further back makes the counting table explode: 3 letters need 38,416 boxes.
• A smarter model (coming in Lesson 5) would need far fewer boxes, but its numbers can't be counted.

So the question is: can your model find good numbers without counting?

Without counting, where would your model even start?

Start random

Instead of counting, your model starts with random scores and fixes them. Row t:

A score for every letter, even ones that never follow t: 14 rows × 14 = 196 scores. Any number, even negative, so easy to nudge.

Learning repeats 4 steps:

1. a)Predict the next character
2. b)Measure how wrong
3. c)Find which way to change each score
4. d)Move each score a little

To predict, your model needs chances, not scores. How do scores become chances?

Scores into chances

Step a) Predict

To guess, your model needs chances that add up to 1, like the rows of the counting table.

Any scores in, chances out. They always add up to 1, and raising one score takes chance away from the others.

This is softmax.🤔

Now your model can guess. But its scores are random, so its guesses are bad. How bad?

How wrong?

Step b) Measure

To get better, your model needs one number that says how bad its guesses are. The text has the right answers. Take its first pair, t → h:

It gave the right answer only 13.3%. How wrong is that?

The less chance for the right answer, the bigger the loss. 100% gives 0.
The rule: loss = −log(chance)🤔

Every pair in the text gets its own loss: 46 pairs, 24 different ones. A repeated pair has the same loss each time, and counts each time:

Try it on new numbers.

✏️ Practice 4.1

New numbers, same ideas.
Remember: loss = −log(chance) (the ln button on a calculator)
The loss of several pairs is the average of their losses.

The goal is to make the loss small. But which of the 196 scores should change, and which way?

Which way?

Steps c) Find directions and d) Move

After t, our text has h 4 times, space 3 times and a full stop once. Start with just one pair, t → h. Your model gave h only 0.133 (13.3%). (The next page uses all 8.)

The rule: gradient = chance − goal
For this pair, h's goal is 1 and every other letter's is 0.
For h: 0.133 − 1 = −0.867🤔

Then move:
new score = old score − gradient
For h: 1.069 − (−0.867) = 1.936, up.
A negative gradient moves a score up, a positive one down, and the further from 0, the bigger the move.

Every other letter goes down, even space and full stop, because this pair says h. On the next page, their own pairs push them up.

In code, loss.backward() finds all 196 gradients at once.

Your turn: fix the scores yourself, using these directions.

You be the model

Now fix row t using all 8 pairs starting with t:

Each pair's goal comes from the text: 1 for its right letter, 0 for the rest. For h, at chance 0.133:

• 4 pairs t → h: 0.133 − 1 = −0.867 (up)
• 4 other pairs: 0.133 − 0 = +0.133 (down)

Averaged: (4 × −0.867 + 4 × 0.133) ÷ 8 = −0.367
= 0.133 − 4/8
4/8 is the share of pairs wanting h: counting's answer, 0.5. The model never sees it; the pairs give it. (Space 3/8, full stop 1/8.)🤔

The loss is the average of the 8 pairs' losses.

Press ▶ Play: below counting's answer the gradient is negative: score up. Above it, down. At it, 0: it stops on the green line.

In code, PyTorch's loss.backward() finds the same gradients. You'll see it on the code page.

Try it on new numbers.

✏️ Practice 4.2

Row x has 4 pairs: x → p, x → p, x → p and x → q. Only p and q exist, and their scores start equal, so each chance is 1/2.
Remember: gradient = chance − goal (goal: 1 for the right letter, 0 for the others)
A row's gradient is the average of its pairs' gradients: add them up, divide by the number of pairs.

That was one row (14 scores) and 8 pairs. Next: all 14 rows, 196 scores and all 46 pairs at once.

Let it learn

So far: one row. Now your model trains all 14 rows at once. One step:

a) Predict: every pair (all 46) gets its chances.
b) Measure: the loss averages the 46 pairs' losses.
c) Find directions: every pair finds its own gradient (chance − goal). Each row averages its own pairs' gradients: one gradient per score.
d) Move every score by that average.
Then it repeats, 300 times.🤔

One new thing: how far to move. The average is multiplied by the learning rate:

• new score = old score − learning rate × gradient
• h in row t starts with gradient −0.367 (the average we found for row t)
• learning rate 1: h goes up 0.367
• learning rate 10: h goes up 3.67

Too small: it creeps. Too big: it jumps past where it should settle, and back.

The same answer, and your model never saw counting's answer. Learning works.🤔

Try it on new numbers.

✏️ Practice 4.3

Remember: new score = old score − learning rate × gradient
gradient = chance − goal
chance = e^score ÷ (sum of all e^score)

The real code averages a little differently.

Averaging all 46

So far, row t averaged over its own 8 pairs. The usual way averages every row over all 46 pairs, the same average the loss already uses.

Row t has 8 of the 46 pairs, so its gradient shrinks by 8/46: h's −0.367 becomes −0.367 × 8/46 = −0.064.

Smaller gradients, so steps are smaller. The fix is a bigger learning rate: the code uses 10. Try both ways.

Averaging over all 46 with learning rate 10 gets there, the same place. Learning rate 1 with it is too slow.

Try it on new numbers.

✏️ Practice 4.4

Row x again: 4 pairs, with 3 followed by p. But now the text has 20 pairs in all, and the average is over all 20.
Remember: gradient = sum of the pairs' gradients ÷ all 20 pairs
new score = old score − learning rate × gradient

Now the code makes sense.

The code

Guess what each print shows. Then press ▶ Run.

import torch
import torch.nn.functional as F

text = "the cat sat on the mat. the dog sat on the log."
unique_chars = sorted(set(text))
vocab_size = len(unique_chars)
char_to_number = {}
number_to_char = {}
for number, char in enumerate(unique_chars):
    char_to_number[char] = number
    number_to_char[number] = char

# Pairs: this character, and the next one
current_chars = []
next_chars = []
for i in range(len(text) - 1):
    current_chars.append(char_to_number[text[i]])
    next_chars.append(char_to_number[text[i + 1]])
current_chars = torch.tensor(current_chars)
next_chars = torch.tensor(next_chars)

# The model: random scores. requires_grad=True lets PyTorch find gradients.
torch.manual_seed(42)
weights = torch.randn((vocab_size, vocab_size), requires_grad=True)

learning_rate = 10
for step in range(300):
    scores = weights[current_chars]                # a) predict
    loss = F.cross_entropy(scores, next_chars)     # b) measure
    weights.grad = None
    loss.backward()                                # c) find directions
    weights.data = weights.data - learning_rate * weights.grad   # d) move
    if step % 50 == 0:
        print("Step", step, "loss:", round(loss.item(), 3))

# What did it learn after 't'?
probabilities = F.softmax(weights, dim=1)
t_row = probabilities[char_to_number['t']]
for char in [' ', '.', 'h']:
    print(repr(char), round(t_row[char_to_number[char]].item(), 3))

$ python lesson4.py
Step 0 loss: 3.086
Step 50 loss: 0.746
Step 100 loss: 0.722
Step 150 loss: 0.714
Step 200 loss: 0.711
Step 250 loss: 0.709
' ' 0.374
'.' 0.124
'h' 0.499

▶ Run it in Google Colab

Colab runs Python with PyTorch in your browser for free. You just need a Google account.

Quick check: did it all make sense?

Lesson 4 test

8 questions, one for each idea in this lesson. Answer them all right to pass. A wrong answer tells you why, and which page to look at again.

What your model still can't do.

Still one letter

Your model learned its numbers by itself. But it still looks at only the last letter.

So it still writes words like “sathe”. To fix that, it must look further back, without the table exploding.

I learned the same chances as counting, all by myself! Now I can find numbers without counting. Can I finally look further back?

Mark lesson as done