GPT-2 reproduce

Andrey Karpathy -> https://www.youtube.com/watch?v=l8pRSuU81PU

And replicate by your self (4hours +)

notes:

PyTorch more user friendly then TensorFlow

gpt2 50257 tokens (vocabulary) X 768 domensions

42 AI