Skip to content

Generative Python Transformer p.4 - Tokenizing

sentdex1.44M subscribers

About this video

Hand-picked by a BrighterTube curator · found via round-3 back catalogue of a channel Lumen picks video by video

Explains tokenization for transformer models, a core preprocessing step for natural language processing. Concrete technique in a methodical series on building generative Python systems.

Video by sentdex, played through YouTube's embedded player.

Part of Generating Python code with GPT