1:55:58Building makemore Part 3: Activations & Gradients, BatchNorm
Andrej Karpathy569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
267,244 videos · Algorithms, systems, security, machine learning and building software.
1:55:58569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
3:31:2410.2M views
A general-audience deep dive into how large language models like ChatGPT are built: pretraining data, tokenisation, neural network internals, post-training and RL.
1:15:40613K views
Makemore part 2: a multilayer perceptron language model following Bengio et al. 2003, with embeddings, minibatches and train/dev/test splits.
4:01:261.23M views
Karpathy reproduces GPT-2 (124M) from scratch: the model, efficient training on GPUs, mixed precision, and evaluation.
56:22305K views
Makemore part 5: building a WaveNet-style convolutional hierarchy and organising PyTorch code into modules.
1:57:45Intro1.32M views
Karpathy builds makemore, a character-level bigram language model, introducing PyTorch tensors, sampling and negative log-likelihood loss.
1:55:24389K views
Makemore part 4: backpropagating manually through a full MLP and batchnorm layer, without autograd.

Intro1.93M views
Lecture 1 of David Silver's reinforcement learning course at UCL: agents, rewards, value functions, policies and the structure of the field.

1.47M views
The first lecture of Yaser Abu-Mostafa's Caltech 'Learning From Data' course: what learning is, the perceptron and the components of a learning problem.

786K views
Andrej Karpathy explains how GPT assistants are trained (pretraining, fine-tuning, RLHF) and how to use them well.
55:57Intro223K views
CS50x 2024, the AI lecture: Artificial intelligence: search, game playing, machine learning and large language models at an introductory level.
1:31:13580K views
A code-first hacker's guide to language models: what they are, how to use them well, fine-tuning and running them locally.
1:30:25166K views
Lesson 3 of Practical Deep Learning (2022): how neural networks and gradient descent actually work, built in a spreadsheet and in code.
2:15:16182K views
Lesson 9 of Deep Learning Foundations to Stable Diffusion: how diffusion models generate images, from the ground up.
1:22:56612K views
Lesson 1 of fast.ai's Practical Deep Learning for Coders (2022): training a working image classifier in minutes and how deep learning got here.
54:512.25M views
Coding a neural network from scratch to recognise doodles: layers, backpropagation and training tricks.
1:16:42247K views
Lesson 2 of Practical Deep Learning (2022): data cleaning, deploying a model and building an app around it.
55:02391K views
Uses floating-point rounding error as the nonlinearity in a neural network, exploring how IEEE floats really behave.

88.4K views
John Schulman explains RLHF: why reinforcement learning helps with truthfulness and hallucination, and the open problems.

274K views
Robert Miles explains instrumental convergence: why almost any goal leads an AI to seek resources and self-preservation.

710K views
Robert Miles explains the orthogonality thesis: why intelligence and goals are independent, so a smart AI need not have 'sensible' goals.
9:40328K views
Nine documented cases where reinforcement-learning systems satisfied the letter but not the intent of their objective.
9:0630.7K views
Part 1 of "Teaching Computers to Learn": what it means for a machine to learn from data, with the history of the idea.
2:26:17628K views
Writes a small machine-learning library in C from scratch: tensors, automatic differentiation and training a network.

838K views
Part 1 of Welch Labs' Neural Networks Demystified: the data and architecture of a simple network, built in Python.
11:2524.4K views
Part 2: the mathematical origins of machine learning, from least squares and regression to fitting models.
16:182.2M views
Learnfun & Playfun: a program that learns to play NES games by watching memory change during human play.
12:2457.6K views
Part 3: Rosenblatt's perceptron, how it learns a linear boundary and where it fails.
44:39523K views
The perceptron, the simplest neural network, implemented from scratch (Nature of Code).
13:35A 14-minute video from Jordan Boyd-Graber: “How to find and measure AI's weak spots [Research, NAACL 2025 Outstanding Paper]”.
47:37A 48-minute video from Simons Institute for the Theory of Computing: “Bridging AI and HCI: Incorporating Human Values into the Development of AI Technologies”.
9:18A 9-minute video from WINDY Lab: “L10: Actor-Critic Methods (P1-The simplest Actor-Critic) —Mathematical Foundations of RL”.
48:32A 49-minute video from Tech With Tim: “Python AI Agent Tutorial - Build a Coding Assistant w/ RAG & LangChain”.
3:06:40A 187-minute video from LxMLS Lisbon Machine Learning School: “Lecture 5: Causality (Adele Ribeiro and Elias Bareinboim)”.
30:56A 31-minute video from Agha Ali Raza - Youtube Channel: “006-02-LogReg_Cost_Function”.
47:02Advanced
A 47-minute video from NAVER LABS Europe: “Visual SLAM and Dynamic Scene Understanding for Autonomous Systems, Daniel Cremers”.