1:55:58Building makemore Part 3: Activations & Gradients, BatchNorm
Andrej Karpathy569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
267,244 videos · Algorithms, systems, security, machine learning and building software.
1:55:58569K views
Makemore part 3: activations, gradients and batch normalisation — diagnosing a network's health from its statistics.
3:31:2410.2M views
A general-audience deep dive into how large language models like ChatGPT are built: pretraining data, tokenisation, neural network internals, post-training and RL.
1:15:40613K views
Makemore part 2: a multilayer perceptron language model following Bengio et al. 2003, with embeddings, minibatches and train/dev/test splits.
4:01:261.23M views
Karpathy reproduces GPT-2 (124M) from scratch: the model, efficient training on GPUs, mixed precision, and evaluation.
56:22305K views
Makemore part 5: building a WaveNet-style convolutional hierarchy and organising PyTorch code into modules.

Intro421K views
Lecture 1A of Abelson and Sussman's 1986 Structure and Interpretation of Computer Programs course: what computer science is about, and an introduction to Lisp.
1:57:45Intro1.32M views
Karpathy builds makemore, a character-level bigram language model, introducing PyTorch tensors, sampling and negative log-likelihood loss.
1:55:24389K views
Makemore part 4: backpropagating manually through a full MLP and batchnorm layer, without autograd.
2:27:41Intro1.41M views
CS50x 2024 lecture 1: an introduction to C — syntax, types, conditionals, loops and compiling.
2:04:55Intro2.2M views
CS50x 2024 lecture 0: how computers represent information, algorithms, and first programs in Scratch.
2:02:12Intro460K views
CS50x 2024 lecture 3: algorithms — searching, sorting, running time and recursion.

Intro629K views
Lecture 1 of MIT 18.404 Theory of Computation by Michael Sipser: finite automata, regular languages and regular expressions.
2:14:05Intro620K views
CS50x 2024 lecture 7: SQL — relational databases, queries, joins and SQL injection.
2:02:49Intro582K views
CS50x 2024 lecture 5: data structures — linked lists, trees, hash tables and tries.
2:17:23Intro399K views
CS50x 2024 lecture 4: memory — pointers, addresses, malloc and the heap.

752K views
MIT's Missing Semester lecture on Git: the data model of blobs, trees, commits and references, then the commands built on it.

Intro1.93M views
Lecture 1 of David Silver's reinforcement learning course at UCL: agents, rewards, value functions, policies and the structure of the field.
41:33199K views
MIT 6.0001, lecture 7: Testing, debugging, exceptions and assertions.
1:22:28Advanced80.4K views
MIT 6.824, lecture 5: Go concurrency patterns and how they apply to implementing Raft.
1:23:44Intro15.3K views
CMU 15-445, lecture 7: Hash tables inside a DBMS: static schemes, linear probing, cuckoo and extendible hashing.
56:54Advanced24.1K views
MIT 6.824, lecture 19: Bitcoin's proof-of-work consensus as a distributed system.
1:19:21Intro6.01K views
CMU 15-445, lecture 20: Write-ahead logging, buffer pool policies and checkpoints.
1:19:0122K views
MIT 6.824, lecture 20: Blockstack and decentralized naming and storage.
1:20:03Advanced35.5K views
MIT 6.824, lecture 11: Cache consistency in the Frangipani distributed file system.
1:20:3280.9K views
MIT 6.824, lecture 8: ZooKeeper's coordination service and its consistency guarantees.
1:22:55Intro4.94K views
CMU 15-445, lecture 24: Distributed analytical databases and query execution across nodes.
51:5794.6K views
MIT 6.006, lecture 5: How direct-access arrays, counting sort and radix sort break the n log n comparison barrier.
57:4036.9K views
MIT 6.006, lecture 12: Bellman-Ford on graphs with negative weights and how it detects negative cycles via graph duplication.
2:28:37Intro246K views
CS50x 2024, lecture 9: Building web applications in Python with Flask, templates and sessions.
1:08:13Advanced30.5K views
MIT 6.824, lecture 15: Spark's resilient distributed datasets for big-data processing.
57:3340.7K views
MIT 6.006, lecture 11: Weighted shortest paths, the relaxation framework and DAG shortest paths.

1.47M views
The first lecture of Yaser Abu-Mostafa's Caltech 'Learning From Data' course: what learning is, the perceptron and the components of a learning problem.
1:23:29Advanced19.1K views
MIT 6.824, lecture 17: Causal consistency in the COPS system.
51:26256K views
MIT 6.0001, lecture 10: Measuring program efficiency and an introduction to big-O.
2:29:29Intro304K views
CS50x 2024, lecture 8: How the web works: HTTP, HTML, CSS and JavaScript.
1:03:4538.4K views
MIT 6.006, lecture 18: Rod cutting, subset sum and what pseudopolynomial running time means.