Skip to content

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (Paper Explained)

About this video

Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by video

Explains GShard's conditional computation and automatic sharding for giant model training. Teaches distributed training techniques for trillion-parameter models.

Video by Yannic Kilcher, played through YouTube's embedded player.

Part of General Machine Learning, Natural Language Processing, Deep Learning Architectures