microgpt

<p>This is a brief guide to my new art project <a href="https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95">microgpt</a>, a single file of 200 lines of pure Python with no dependencies that trains and inferences a GPT. This file contains the full algorithmic content of what is needed: dataset of documents, tokenizer, autograd engine, a GPT-2-like neural network architecture, the Adam optimizer, training loop, and inference loop. Everything else is just efficiency. I cannot simplif