How Language Models Work

Lesson 05

Training

Nobody writes the rules. The misses do.

Cover up the next word, make the model guess, measure how wrong it was, nudge every dial toward less wrong. Now repeat a few trillion times.

The cat sat on the  

    Loss: how wrong the guesses are, on average. Down and to the right is learning.