Zml-smi: universal monitoring tool for GPUs, TPUs and NPUs (opens in new tab)

(zml.ai)

77 pointssteeve1mo ago11 comments

11 comments

Looks cool!

nvtop can actually support TPUs too via https://github.com/rdyro/libtpuinfo/ https://github.com/Syllo/nvtop/blob/76890233d759199f50ad3bdb...

serialx1mo ago

Look into all-smi https://github.com/lablup/all-smi It supports all GPUs thinkable including Apple Silicon and many AI accelerator cards.

mrflop1mo ago

Renaming fopen64 to intercept library calls feels like a brittle hack masquerading as "sandboxing." Why not just upstream this hardware support to nvtop instead of fragmenting the ecosystem?

steeveOP1mo ago

sadly, sandboxing is something that can't be upstreamed. this way, sandboxing is kept in zml instead of patching mesa.

as for nvtop, great program, but we missed a few features (such as sandboxing)

pstuart1mo ago

It looks cool and I was excited to get monitoring for the NPU on my Ryzen AI 395+, unfortunately it does not show. NPU support in linux really seems to be an afterthought.

steeveOP1mo ago

Weird, because we tried it. It doesn’t show anything?

We use the amdsmi to get metrics. I’ll investigate.

marwanet1mo ago

If this logic were pushed into nvtop, wouldn't the codebase become unmaintainable? Each vendor's interception method is going to be different.

1 more reply

imcritic1mo ago

Is it capable of exposing metrics in Prometheus format?

steeveOP1mo ago

consider it done

152334H1mo ago

"NPU" seems to refer to trainium only?

synergy201mo ago

would be nice to have cpu usage added so I have all in one?

currently I use btop which shows basic gpu load along with cpu, network, etc.

j / k navigate · click thread line to collapse

11 comments

rdyro1mo ago

Looks cool!

nvtop can actually support TPUs too via https://github.com/rdyro/libtpuinfo/ https://github.com/Syllo/nvtop/blob/76890233d759199f50ad3bdb...

serialx1mo ago

Look into all-smi https://github.com/lablup/all-smi It supports all GPUs thinkable including Apple Silicon and many AI accelerator cards.

mrflop1mo ago

Renaming fopen64 to intercept library calls feels like a brittle hack masquerading as "sandboxing." Why not just upstream this hardware support to nvtop instead of fragmenting the ecosystem?

steeveOP1mo ago

sadly, sandboxing is something that can't be upstreamed. this way, sandboxing is kept in zml instead of patching mesa.

as for nvtop, great program, but we missed a few features (such as sandboxing)

pstuart1mo ago

It looks cool and I was excited to get monitoring for the NPU on my Ryzen AI 395+, unfortunately it does not show. NPU support in linux really seems to be an afterthought.

steeveOP1mo ago

Weird, because we tried it. It doesn’t show anything?

We use the amdsmi to get metrics. I’ll investigate.

marwanet1mo ago

If this logic were pushed into nvtop, wouldn't the codebase become unmaintainable? Each vendor's interception method is going to be different.

1 more reply

imcritic1mo ago

Is it capable of exposing metrics in Prometheus format?

steeveOP1mo ago

consider it done

152334H1mo ago

"NPU" seems to refer to trainium only?

synergy201mo ago

would be nice to have cpu usage added so I have all in one?

currently I use btop which shows basic gpu load along with cpu, network, etc.

j / k navigate · click thread line to collapse