undefined | Better HN

0 pointstrashtester1y ago0 comments

All of physics basically reduces to linear algebra locally (which becomes quite unilinear when enough tensors are multiplied).

Why shouldn't we expect AI to be created using the same type of math?

If there is a surprise, it's only that we can use the same math at a much higher level of abstraction than the quantum level.

0 comments

nerdponx1y ago

That's a really nice analogy, I wonder if there is some research on this kind of thing in the machine learning literature already.

trashtesterOP1y ago

There probably is, since I believe tensors were basically borrowed from Physics at some point. But it's probably not of much practical use today, unless you want to explore Penrose's ideas about microtubules or something similarly exotic.

Gains in AI and compute can probably be be brought back to physics and chemistry to do various computations, though, and not limited to only protein folding, which is the most famous use case now.

nerdponx1y ago

For what it's worth, the idea of a "tensor" in ML is pretty far removed from any physical concept. I don't know its mathematical origins (would be interesting I'm sure), but in ML they're only involved because that's our framework for dealing with multi-linear transformations.

Most NNs work by something akin to "(multi-)linear vector transformation, followed by elementwise nonlinear transformation", stacked over and over so that the output of one layer becomes the input of the next. This applies equally well to simple models like "fully-connected" / "feed-forward" networks (aka "multi-layer perceptron") and to more-sophisticated models like transformers (e.g. https://github.com/karpathy/nanoGPT/blob/325be85d9be8c81b436...).

It's less about combining lots of tiny local linear transformations piecewise, and more about layering linear and non-linear transformations on top of each other.

I don't really know how physics works beyond whatever Newtonian mechanics I learned in high school. But unless the underlying math is similar, then I'm hesitant to run too far with the analogy.

2 more replies

j / k navigate · click thread line to collapse

0 comments

nerdponx1y ago

That's a really nice analogy, I wonder if there is some research on this kind of thing in the machine learning literature already.

trashtesterOP1y ago

Gains in AI and compute can probably be be brought back to physics and chemistry to do various computations, though, and not limited to only protein folding, which is the most famous use case now.

nerdponx1y ago

It's less about combining lots of tiny local linear transformations piecewise, and more about layering linear and non-linear transformations on top of each other.

I don't really know how physics works beyond whatever Newtonian mechanics I learned in high school. But unless the underlying math is similar, then I'm hesitant to run too far with the analogy.

2 more replies

j / k navigate · click thread line to collapse