Skip to content

ALiBi - Train Short, Test Long: Attention with linear biases enables input length extrapolation

About this video

Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by video

Explains ALiBi attention mechanism enabling length extrapolation. Teaches technique to train on short sequences and test on longer ones.

Video by Yannic Kilcher, played through YouTube's embedded player.

Part of Natural Language Processing, Papers Explained