Welcome to Technical AI

Hi, I’m Lorenzo and I’m the author of the Technical AI blog, where I share ML/AI content for free.

Title	Reading Time
How softmax is implemented in practice	4 min
Why we need to divide by \(\sqrt{d_k}\) in attention?	3 min
Transformers in a nutshell	2 min
(4/6) AI in Multiple GPUs: Grad Accum & Data Parallelism	16 min
(3/6) AI in Multiple GPUs: How GPUs Communicate	5 min
(2/6) AI in Multiple GPUs: Point-to-Point and Collective Operations	11 min
(1/6) AI in Multiple GPUs: Understanding the Host and Device Paradigm	7 min
RoPE clearly explained	9 min