undefined | Better HN

story

0 pointsFridgeSeal6y ago0 comments

Kubeflow, unless I’m missing some things, is for Tensorflow pipelines, if you’re not using TF, or it’s not the only thing you use, it’s not ideal.

I thought MlFlow was a spark thing, and were trying to migrate off of spark/DataBricks due to the resources inefficiencies of Spark (at our scale) and maintenance nightmare that python notebooks are causing us.

Argo is just a container workflow tool, not ML specific. We’re planning on using Argo for the data engineering parts, and polyaxon for the ML training parts because of the convenient monitoring and hyper parameter search tools.

0 comments

TheIronYuppie6y ago

Hi! Co-founder of Kubeflow here - definitely not TensorFlow only! You can see ([1]) many many different repos and operators. The nice part about Argo for us is it let us build an ML specific DSL that was also Kubernetes native.

[1] https://github.com/kubeflow/

j / k navigate · click thread line to collapse

0 comments

TheIronYuppie6y ago

[1] https://github.com/kubeflow/

j / k navigate · click thread line to collapse