danielEM

If you're a software developer and using coding agents "all the time" for your work please share here your bills and APIs you're working with.

I was easily able to burn $1 a minute with deepinfra. But I have just tried it only a couple of times.

Can't find "apple to apple" comparison on performance on QWQ 32b (4bit), can anyone help me with decision on which solution to pick?

From what I dig so far it looks like dual Arc A770 is supported by llama.cpp. And saw some reports that llama.cpp on top of IPEX-LLM is fastest way for inference on intel card.

On the other end there is more expensive 7900 XTX on which AMD claims (Jan '25) that inference is faster than on 4090.

So - what is the state of the art as of today, how does one compare to another (apple to apple)? What is tokens/s diff?

Is AI gonna take over these jobs? What tools to use? Where to head first steps? Any volunteering opportunities in that area?

For what purposes you use llama.cpp?

What tools do you use with llama.cpp?

Is there anything you recommend to avoid when it comes to llama.cpp?

Want to collect your best practices/experiences and advice around llama.cpp. Eg. if you work with Visual Studio Code - what plugins you recommend, and what not. Etc...

Personally having an issue like this one:

[ 6488.577727] [drm:amdgpu_job_timedout [amdgpu]] ERROR ring gfx_0.0.0 timeout, signaled seq=1715593, emitted seq=1715595

[ 6488.577881] [drm:amdgpu_job_timedout [amdgpu]] ERROR Process information: process firefox pid 756 thread firefox:cs0 pid 831

[ 6488.578005] amdgpu 0000:0d:00.0: amdgpu: GPU reset begin!

And wondering - is it linux driver or is it firefox? Anyone have any experience/insights?