Attention

Every word gets to look at every other word.

Before each guess, the model scores how much every earlier word matters right now. That scoring is called attention. Watch one little word switch teams.