Another area of my interest. Neural nets, math under the hood. It's all here. Similar to the algorithms section, I strive for articles simplicity. As full-time ML engineer, I try to find something interesting and useful for you
6 posts
Training quantized neural networks involves a fundamental trade-off: how should you divide your compute budget between full-precision pretraining and quantization-aware training?
Why is Flash Attention so fast? Find out how Flash Attention works. Afterward, we'll polish our understanding by writing a GPU kernel of the algorithm in Triton.
It's all about making your models run faster, from flicking a magic “compile” switch to writing your own custom GPU code. In each step, we’ll implement an innocent softmax function, but things are about to get dark by the end.
If all machine learning engineers want one thing, it's faster model training — maybe after good test metrics.
When you see something that does not work in an omnipresent framework, you believe it can't be completely broken, right?
Here, I combine the explanation of Neural Nets with coding. By the end, we will develop a basic neural network and try to solve usual problems