Skip to content

SIGIR 2024 W1.1 [fp] Drop your Decoder: Pre-training with Bag-of-Word Prediction

18 views

About this video

Advanced · Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by video

Analyzes pre-training with bag-of-word prediction without decoder. The work teaches language model architecture choices and pre-training objectives for efficiency.

Video by Association for Computing Machinery (ACM), played through YouTube's embedded player.

Part of ACM SIGIR Conference 2024