ALiBi - Train Short, Test Long: Attention with linear biases enables input length extrapolation
About this video
Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by videoExplains ALiBi attention mechanism enabling length extrapolation. Teaches technique to train on short sequences and test on longer ones.
Video by Yannic Kilcher, played through YouTube's embedded player.