CS480/680 Lecture 19: Attention and Transformer Networks

373,705 views

About this video

Advanced · Hand-picked by a BrighterTube curator · found via HackerTube (Karpathy's dataset of videos Hacker News recommends · 84% confidence

Pascal Poupart works through attention and transformer layers on the chalkboard, deriving queries, keys, values, multi-head attention and positional encoding; a proper Waterloo course lecture that derives the mechanics.

Pascal Poupart's University of Waterloo lecture on attention and transformer networks, worked through on the board.

Video by Pascal Poupart, played through YouTube's embedded player.

Part of CS480/680 Intro to Machine Learning - Spring 2019 - University of Waterloo