Well, first step is to run it. I am getting a step every 3 seconds. Is that typical?i7-13700H, 2400 Mhz, 14 Core(s), 20 Logical Processor(s). It is using 12 CPUs which is good. And 160Mb memory.

Hey! Yes please dive in :) It can be a starting point for learning Rust ;)
MatMul is fine, the bottleneck is LayerNorm + Softmax gradients right now. Haven&#x27;t been able to optimize them well yet.

Well done! I would be crazy if I dived into this having never studied Rust! But it looks very interesting so I might do that! I have almost finished AK’s lecture. Have you made the matmul faster yet? That would seem a priority?

I sure hope this is the unit of measurement and not a Berserk reference

Show HN: FemtoGPT – Pure Rust implementation of a GPT language model