From Andrey Karpathy MicroGPT
This small project can train a GPT-2 like neural network and also do inference. It has only about 200 lines of Python code and no dependencies.
Inside, it includes everything:
- dataset
- tokenizer
- engine
- training
- inference
As Andrej says, it is probably hard to simplify this more.
He made earlier projects like nanoGPT that i played before. With MicroGPT, he reduced the idea to the bare minimum.
It’s a good project for people who want to understand how GPT works in a simple way.
https://karpathy.github.io/2026/02/12/microgpt/ karpathy.github.io/2026/02/12/microgpt/