1:55:58Building makemore Part 3: Activations & Gradients, BatchNorm
Andrej Karpathy569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
267,244 videos · Algorithms, systems, security, machine learning and building software.
1:55:58569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
3:31:2410.2M views
A general-audience deep dive into how large language models like ChatGPT are built: pretraining data, tokenisation, neural network internals, post-training and RL.
1:15:40613K views
Makemore part 2: a multilayer perceptron language model following Bengio et al. 2003, with embeddings, minibatches and train/dev/test splits.
4:01:261.23M views
Karpathy reproduces GPT-2 (124M) from scratch: the model, efficient training on GPUs, mixed precision, and evaluation.
56:22305K views
Makemore part 5: building a WaveNet-style convolutional hierarchy and organising PyTorch code into modules.
1:57:45Intro1.32M views
Karpathy builds makemore, a character-level bigram language model, introducing PyTorch tensors, sampling and negative log-likelihood loss.
1:55:24389K views
Makemore part 4: backpropagating manually through a full MLP and batchnorm layer, without autograd.

Intro1.93M views
Lecture 1 of David Silver's reinforcement learning course at UCL: agents, rewards, value functions, policies and the structure of the field.

1.47M views
The first lecture of Yaser Abu-Mostafa's Caltech 'Learning From Data' course: what learning is, the perceptron and the components of a learning problem.
55:57Intro223K views
CS50x 2024, the AI lecture: Artificial intelligence: search, game playing, machine learning and large language models at an introductory level.
55:02391K views
Uses floating-point rounding error as the nonlinearity in a neural network, exploring how IEEE floats really behave.
30:56
17:47
17:35
50:38
15:17
1:33:39
24:51
1:15:24
34:31
11:46
1:45:15
47:39
1:08:14
1:13:40
1:13:45
21:12
9:20
1:21:30
11:13
2:28:18
48:33
13:14
52:53
13:51
5:14