undefined | Better HN

0 pointsjeffparsons3y ago0 comments

Question from an ML-illiterate:

Are there known ways the training for these models could be distributed/decomposed? E.g. SETI-style distribution of a homogenous centrally-defined task, or — much more exciting — recombination of several different models / sets of weights? (I'm just throwing words around here without really understanding them.)

I'm imaging a world in which one group of enthusiasts could work together to train a model on all images on Wikipedia, another group could work on training a model that understands hands really well, and then later yet another group could combine the work of the other two without doing all that training from scratch.

Is that even remotely plausible?

0 comments

3 comments · 3 top-level

wokwokwok3y ago

It’s almost certainly not worth the bother.

The effort and time involved in setting up a distributed community training system would be extremely prone to abuse, errors and uncertainty about the results.

You could get better quality, more quickly by simply running a kickstarter and paying for dedicated gpu time.

swyx3y ago

not sure how illiterate you are, you're asking good questions, but fwiw if you watch the corridor digital video you should be able to grasp how much transfer learning is possible https://www.youtube.com/watch?v=W4Mcuh38wyM

sophrocyne3y ago

Yes it is!

j / k navigate · click thread line to collapse