Lightweight Model Serving (opens in new tab)

(github.com)

1 pointsmartynassubo1y ago1 comments

1 comments

Hello,

I recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.

Here are the findings:

https://github.com/martynas-subonis/model-serving

Additionally, I wrote a broader article on Substack for anyone interested:

https://github.com/martynas-subonis/model-serving

Hope somebody finds it interesting/useful as well.

j / k navigate · click thread line to collapse

1 comments

martynassuboOP1y ago

Hello,

Here are the findings:

https://github.com/martynas-subonis/model-serving

Additionally, I wrote a broader article on Substack for anyone interested:

https://github.com/martynas-subonis/model-serving

Hope somebody finds it interesting/useful as well.

j / k navigate · click thread line to collapse