ArangoML Pipeline: Common Metadata Layer for Machine Learning

noah-kunOP6y ago· 2 in thread

Arango is doing so much right now--their SaaS version "Oasis" coming out this year, SmartJoins for faster multi-cluster joins and now ML.

I admit I liked Mongo for it's document storage. But I didn't like Postgres due to my irrational dislike of SQL Then I challenged myself to try out a graph database.

ArangoDB presents things like Mongo (as documents), has a much nicer query language than SQL (or just use GaphQL) and does graph stuff; all while being amongst the top 3 or so in benchmarks that are designed to test only specific types of databases. If databases did triathlons, ArangoDB would be unbeatable.

Only two issues that I struggled with are/were Kubernetes cluster stuff, like sharding, but this is common to cluster DBs I think.

joerg846y ago

Did you try the Kubernetes Operator: https://github.com/arangodb/kube-arangodb? Should at least help with the k8s part... Also would be curious about your sharding challenges, feel free to ping me on the ArangoDB slack

noah-kunOP6y ago

Yes, I struggled with the operator, to the point where it would not install at all. I even fixed some of the documentation, but later my database was not responsive.

I ended up using the somewhat deprecated community-made .yaml files in https://github.com/sbaugher/arangodb-kubernetes

It has a problem of it's own, but I have some idea how to fix it (next time it happens), and am very interested in Oasis.

joerg846y ago

One of the authors here: Feel free to checkout the meetup talk https://www.meetup.com/Knowledge-Graphs-Meetup/events/264962... and the corresponding slides: https://docs.google.com/presentation/d/1r4rLhjoDN_CWpjMUz29s...

janemanos6y ago

Here is the repo on Github https://github.com/arangoml/arangopipe

ArangoML Pipeline: Common Metadata Layer for Machine Learning (opens in new tab)

ArangoML Pipeline: Common Metadata Layer for Machine Learning (opens in new tab)

5 comments

5 comments