How do context clues help predict the next word in a sentence?

ChatGPT changed the game by being able to write long paragraphs that make sense. Why is this impressive?

Let's use an attention model to generate new song lyrics!

Learning about the Attention Mechanism Educator Guide | 2022-23 Day of AI Slide Show

Make your own copy of these slides

Session recording

California K-12 CS Standards

9-12.DA.11 Refine computational models to better represent the relationships among different elements of data collected from a phenomenon or process.

AI4K12 Big Ideas in AI

9-12.Representation 2-A-iv Describe how a transformer network operates.

Structure of Learning Activity

Introducing the Attention Mechanism
Training Algorithm
Testing & Evaluating Algorithm

Intended to be completable in a 1 hour class, though with iterative refinement it could go longer.

Materials

In-Class Slide Deck, Educator Guide, as well as Replit & GitHub links provided in the posted slide deck above!

Page updated

Google Sites

Report abuse