Bringing Up DeepSeek-V4-Flash on AMD MI300X (opens in new tab)

(fergusfinn.com)

120 pointskkm23d ago25 comments

25 comments

17 comments · 7 top-level

mezark23d ago· 4 in thread

We at doubleword are bullish for AMD for low-interactivity inference - it does just take a bigger lift on the software side...

brcmthrowaway22d ago

Are you long AMD?

latchkey22d ago

Interesting that you ask that as AMD hits another ATH.

brcmthrowaway22d ago

Then you are definitely long on AMD.

1 more reply

boxking22d ago

hello,sir, I want bulk order Asrock BC-250, is it still available ?

1 more reply

edg500022d ago· 2 in thread

Checked out this company about a year ago and they only offered small models. Now I see they have GLM-fp8/Kimi and DeepSeek V4 Pro. Since workloads are predominantly cached input, I'm surprised to see no separate price for cached input vs uncached. I hope the prices will drop significantly; with these prices you'll end up with thousands in monthly costs quickly. Hopefully more hardware companies will be on the market in the coming years. If the Chinese eventually start competing with the current memory makers, maybe that will help.

mezark22d ago

Hi! Co-founder of Doubleword here - we've hugely increased the number of models that we offer (partly thanks to work that we've done on hotswapping https://blog.doubleword.ai/fast-sglang-starts.

We're kind of known for our low prices - our prices (our main usage is for our high throughput API - the async tier) is significantly below average openrouter prices - but cached prices is coming soon which will lower them even more :)

edg500019d ago

What kind of workloads are you primarily seeing from users? I´d guess coding harness-type stuff where you have repeated calls with lots of cache hits. Or is it more like bulk OCR or invoice processing?

latchkey22d ago· 2 in thread

Nice work and thanks for being a customer.

(CEO Hot Aisle)

erichocean22d ago

I wish you guys could partner with Modular to get Mojo inference working on your hardware, e.g. https://www.modular.com/models/deepseek-v4-pro

latchkey22d ago

Not sure I understand. If they support MI300x, their self-hosted will run on our hardware.

1 more reply

maCDzP22d ago· 1 in thread

I train on AMD MI250X and managed to get Gemma 4 31B to work - but it took a lot of work on the software side.

kkmOP22d ago

This is very interesting, planning to write about it?

benlm23d ago· 1 in thread

Nice work! Would DeepSeek V4 Pro on 8xMI300X work with these patches?

mezark22d ago

we think so - but haven't tested it ourselves

kkmOP23d ago

Also the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword

alfiedotwtf22d ago

It’s just weird Deepseek released a model that was not compatible with any of the usual engines. Without derez’s new project just to support DSv4, how long until it’s actually viable in llama :(

j / k navigate · click thread line to collapse

25 comments

17 comments · 7 top-level

mezark23d ago· 4 in thread

We at doubleword are bullish for AMD for low-interactivity inference - it does just take a bigger lift on the software side...

brcmthrowaway22d ago

Are you long AMD?

latchkey22d ago

Interesting that you ask that as AMD hits another ATH.

brcmthrowaway22d ago

Then you are definitely long on AMD.

1 more reply

boxking22d ago

hello,sir, I want bulk order Asrock BC-250, is it still available ?

1 more reply

edg500022d ago· 2 in thread

mezark22d ago

Hi! Co-founder of Doubleword here - we've hugely increased the number of models that we offer (partly thanks to work that we've done on hotswapping https://blog.doubleword.ai/fast-sglang-starts.

edg500019d ago

latchkey22d ago· 2 in thread

Nice work and thanks for being a customer.

(CEO Hot Aisle)

erichocean22d ago

I wish you guys could partner with Modular to get Mojo inference working on your hardware, e.g. https://www.modular.com/models/deepseek-v4-pro

latchkey22d ago

Not sure I understand. If they support MI300x, their self-hosted will run on our hardware.

1 more reply

maCDzP22d ago· 1 in thread

I train on AMD MI250X and managed to get Gemma 4 31B to work - but it took a lot of work on the software side.

kkmOP22d ago

This is very interesting, planning to write about it?

benlm23d ago· 1 in thread

Nice work! Would DeepSeek V4 Pro on 8xMI300X work with these patches?

mezark22d ago

we think so - but haven't tested it ourselves

kkmOP23d ago

Also the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword

alfiedotwtf22d ago

j / k navigate · click thread line to collapse