Ask HN: What's something interesting you learned from training your own GPT?

Not using APIs, actually training a model from scratch, even a small one

What surprised you about the data, the training process, or the output?