I am working on this at the moment, and no, I don't expect that what I write here will convince anyone. And I don't have any summaries of the work at the moment.
It's clear that there is a relationship between computation and information via Landauer's principle. It's also clear that it's got to do with nonlinearity of dynamical systems: "The essence of computation is nonlinear logical operations." J. Hopfield, PNAS 79, 2554 (1982).
Semantics: in short, 'meaning' is just a mapping (ie, mutual information) between language and the physical world. https://heteroskedasticblog.wordpress.com/2018/01/13/informa...
Could you elaborate? It's not clear to me what you mean by that.
> It's clear that there is a relationship between computation and information via Landauer's principle.
Information, in the sense I'm concerned with, is an inherent part of computation. You can't have the latter without the former.
> Semantics: in short, 'meaning' is just a mapping (ie, mutual information) between language and the physical world.
That doesn't explain meaning in the case of imaginary or abstract details, or the system's conception of the meaning.
.
BTW, would I be able to get a copy of that Hopfield paper? I wasn't able to find a copy of it.
OTOH, the algorithmic complexity theory (a la Kolmogorov) doesn't really have the same generality as Shannon's theory. Flops is not a well-defined measure of information processing rate for the brain, for instance.
I got inspired by the "integrated information theory" folks -- they have this notion that combining information streams in a nontrivial* way is necessary and sufficient for consciousness. I disagree that it's sufficient for consciousness, but it might be sufficient for a definition of information processing or generalized computation.
> You can't have the latter without the former.
Agreed.
>That doesn't explain meaning in the case of imaginary or abstract details, or the system's conception of the meaning.
The mapping is in our heads. I don't know what you mean by "the system's conception of meaning" -- which system, and what is a conception of meaning?
Here's a link to the paper: https://www.pnas.org/content/pnas/79/8/2554.full.pdf
edit: *how to define 'nontrivial' is very much up for debate. edit2: formatting
In my view, it is more than a change of representation. The computation is using information that might true of something, to produce new information that might be true of something else. I don't see how anything like Shannon's theory could explain how it is able to do this.
> I got inspired by the "integrated information theory" folks -- they have this notion that combining information streams in a nontrivial way is necessary and sufficient for consciousness. I disagree that it's sufficient for consciousness, but it might be sufficient for a definition of information processing or generalized computation.
Ok. I share the same view, that it isn't sufficient for consciousness.
> I don't know what you mean by "the system's conception of meaning" -- which system, and what is a conception of meaning?
The computational system. Consider the case of the human brain, which may be computational. People can understand that some information is about X (say, a particular tree, or the notion of Justice). But it's not just that they know what the information is about, but they understand something of the character of that thing -- of the tree, or of what Justice is like. If the brain is computational, then that would mean that such an understanding was computational (or computational plus bodily interactions with the environment, etc). But that doesn't tell us how it is that computation is able to "embody" an understanding of the character of something. That needs to be explained.
> Here's a link to the paper
Thank you.
Is that the same paper? I notice it has a different title to the one you mentioned above.
As is well known, information theory doesn't deal with the meaning of the messages. As far as information goes, I am primarily concerned with information in the sense of such "messages", and their meaning.
Computation is considered by many to not involve semantics, because it processes information in a "blind, pattern matching" fashion, without regard to its potential meaning. But the sense in which it is semantic does not have to do with its intrinsic characteristics. It it a matter of its "extrinsic" details - how the information states in it relate to details that are (typically) outside of the computation. Seeing the computation as a physical process, processing information "about" details in its environment, highlights this.
The key to all this is appreciating that, once you see semantics and information processing as a matter of physical processes, you can see that the correct semantics can be necessary for producing a physical outcome. So you can analyse how that physical outcome was produced, in order to understand the semantics.
