Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
Understanding Multi-Head Latent Attention (From DeepSeek)
(opens in new tab)
(shreyansh26.github.io)
2 points
shreyansh26
5mo ago
1 comments
Save
Share
1 comments
1 comments · 1 top-level
top
newest
oldest
shreyansh26
OP
5mo ago
A short deep-dive on Multi-Head Latent Attention (MLA) (from DeepSeek): intuition + math, then a walk from MHA → GQA → MQA → MLA, with PyTorch code and the fusion/absorption optimizations for KV-cache efficiency.
j
/
k
navigate · click thread line to collapse
