undefined | Better HN

0 pointsGregarianChild2y ago0 comments

Can you point me to papers with reproducible benchmarking that achieves big speedups on those?

Modern GPUs are GP-GPUs: where GP means "general purpose": you can run any code on GPGPUs. But if you want to gain real speed-ups you will have to program in an awkward style ("data parallel"). I am not aware of GPU acceleration of the work-horses of symbolic AI, such as Prolog, or SMT solving. There has been a lot of work on running SAT-solvers on GPUs, but I don't think this has really succeeded so far.

0 comments

claytonjy2y ago

I think we're conflating two things: shallow/classic ML is not symbolic AI. I'm not sure "ML" even encompasses anything "symbolic"; I see symbolic AI and ML as subfields with little overlap.

I'm not saying symbolic AI has been GPU accelerated in the past, but that non-deep ML has been.

saltcured2y ago

Back when I took AI courses in the early 90s, ML was anything that was trained by data. It did not refer exclusively to dense numerical or statistical methods. It included decision trees, which I think of as being closer to the symbolic camp...

GregarianChildOP2y ago

There is no agreement on the exact meaning of ML and AI. They are often used interchangeably. And for good reason, because it's all about getting computers to learn. We should not squabble about semantics.

Can you point towards papers that report substantial GPU acceleration on what you call "non-deep ML"?

claytonjy2y ago

You're certainly right that AI/ML is often used interchangeably, though I think the distinction is pretty clear among practitioners: AI is the big circle, ML is a subset of AI, deep learning is a subset of ML. Symbolic AI is also a subset of AI, though I'm not familiar enough with it to say how much it interects the others.

So ML is AI, but just a small part of it.

As for papers, here's one showing GPU speedups for the 3 big GBM packages: https://arxiv.org/pdf/1809.04559.pdf. Their setup shows a 3-7x speedup for XGBoost.

chelmney2y ago

Let‘s not forget Gaussian processes: https://gpytorch.ai/, https://www.kernel-operations.io/keops/index.html Some of the work I‘ve been doing in this field only became feasible in a reasonable amount of time due to the use of GPUs.

fancyfredbot2y ago

The parent comment is talking about SVM, which is not a form of symbolic AI. SVM and other kernel based algorithms contain naturally data parallel steps like summing f(z-x[i]) for some kernel function f and all sample data x[i]. These will work great on GPU once you have a few thousand data points.

j / k navigate · click thread line to collapse

0 pointsGregarianChild2y ago0 comments

Can you point me to papers with reproducible benchmarking that achieves big speedups on those?

0 comments

claytonjy2y ago

I think we're conflating two things: shallow/classic ML is not symbolic AI. I'm not sure "ML" even encompasses anything "symbolic"; I see symbolic AI and ML as subfields with little overlap.

I'm not saying symbolic AI has been GPU accelerated in the past, but that non-deep ML has been.

saltcured2y ago

GregarianChildOP2y ago

Can you point towards papers that report substantial GPU acceleration on what you call "non-deep ML"?

claytonjy2y ago

So ML is AI, but just a small part of it.

As for papers, here's one showing GPU speedups for the 3 big GBM packages: https://arxiv.org/pdf/1809.04559.pdf. Their setup shows a 3-7x speedup for XGBoost.

chelmney2y ago

fancyfredbot2y ago

j / k navigate · click thread line to collapse