undefined | Better HN

0 pointsrfoo1y ago0 comments

For FlashMLA? No. The code here runs on one GPU only and do not have a builtin communication part.

0 comments

1 comments · 1 top-level

But for the training it does. You need to communicate gradient changes between GPUs.

j / k navigate · click thread line to collapse