Jagged Flash Attention Optimization (opens in new tab)

(shaped.ai)

24 pointstullie1y ago3 comments

3 comments

Flash attention natively supports packing multiple variable length sequences into a single call, what is the advantage of jagged flash attention?

If only there was a link to a page somewhere that could answer this question for you.

Flash attention natively supports packing multiple variable length sequences into a single call, what is the advantage of jagged flash attention?

If only there was a link to a page somewhere that could answer this question for you.