Serving ML at the Speed of Rust (opens in new tab)

(shvbsle.in)

21 pointsdropbox_miner3y ago2 comments

2 comments

> Rust is not mature enough to support ML out of the box ... If you want to serve your Xgboost or deep learning model then Rust is not the right choice (yet).

Would have liked to hear more about the parenthetical 'yet' even if its just links to other blog posts.

minimaxir3y ago

As the article notes, there isn't any official Rust-native support for any common frameworks, and using a C wrapper limits its advantages.

tract (https://github.com/sonos/tract) seems like the most mature for ONNX (for which TF/PT export is good nowadays), and recently it successfully implemented BERT via Hugging Face ONNX export.

j / k navigate · click thread line to collapse

2 comments

occupant3y ago

> Rust is not mature enough to support ML out of the box ... If you want to serve your Xgboost or deep learning model then Rust is not the right choice (yet).

Would have liked to hear more about the parenthetical 'yet' even if its just links to other blog posts.

minimaxir3y ago

As the article notes, there isn't any official Rust-native support for any common frameworks, and using a C wrapper limits its advantages.

tract (https://github.com/sonos/tract) seems like the most mature for ONNX (for which TF/PT export is good nowadays), and recently it successfully implemented BERT via Hugging Face ONNX export.

j / k navigate · click thread line to collapse