Growing Neural Cellular Automata: A Differentiable Model of Morphogenesis (opens in new tab)

We have a lot of future applications and directions in mind, including the feasibility of applying this to distributed computing and consensus. In particular, the idea of training a model generating CA parameters instead of having a CA fit a specific pattern is of great interest to us and the synthetic biological field. Feel free to get in touch.

This is a wonderful idea. Would you be interested in collaborating on this project?

Diffusion-reaction systems were an inspiration in general. Using a Laplace operator (or the discretised equivalent for our 2D grid) might have trouble learning to generate these patterns - the Laplacian wouldn’t always provide unique information as to where a cell might be based on its neighbours. It’s possible the network would learn to exploit the hidden channels to bypass this directional invariance. Starting from a single pixel in such a setting would indeed need some mechanism to break the symmetry (stochastic updates as used here, for instance).

Now having scrolled down to the bottom of the papaer, I have seen a section on the genesis of the idea.

1) This is often the first question many people familiar with NNs ask and rightly so. Compression was not one of our goals with this article, and it would look like a terrible compression algorithm if that were its purpose. In fact, the model displayed here is about 8.3k parameters, although the WebGL model is quantized (more info on this in the last section), and each model learns to encode an image consisting of 44x44x3 = 5808 integers. We made no attempt to minimize this number. The key thing to bear in mind is that all the cells have the exact same rule and the image generation starts from a single one of them - meaning they have to learn to communicate locally with their neighbours to self-organize in the correct pattern. This is a very non-trivial task and the majority of the model’s parameters are likely going towards this communication protocol and growth behavior.

2) We have not tried 3D. As for animations, some of our earliest experiments suggested one could achieve “animations” by applying the loss at key-points to have the model learn to iterate through these points across several time steps.

3) One could argue the WebGL implementation does this to some extent by quantizing the learned weights we take from the Tensorflow training code. The model remains very resilient and worked out of the box in almost all cases. Moreover, if one tried to inject explicit noise to the CA in a given location, some models would have no problems adapting to it, while others would fail miserably. Some early experiments yielded some remarkably resistant models, able to resist while being subject to continuous globally occurring noise. We suspect explicitly training them while introducing noise would allow us to drive the model towards more consistently resistant behaviors.

4) One of the main obstacles to larger patterns at the moment is memory usage during a forward/backward pass. There are optimization and tricks we plan to employ to generate larger and more complex patterns, which may be discussed in a follow up thread.

I'm wondering these things too. You say in the paper that your net is quite small - I wonder if you could first extend it to regenerative sprite animations for some fun procedural NPCs in a roguelike? If you can efficiently add or change a pixel or two per frame in a sprite+collision mask, imagine battle sequences where things get hacked off, maybe regrow if you wait too long, etc.

1. The results do not depend on speed in principle. The difference between 60FPS and “max” can be several orders of magnitude depending on your hardware, so it’s likely you may see instabilities that exist on much longer horizons at higher speeds.

Interactions will play out very differently at different speeds because you are interacting with a sped-up/slowed down version of the CA.

2. We haven’t done any rigorous studies of regenerative capability, although this is certainly on our to-do list. From empirically playing with them, models seem to be more susceptible to damage to the centre of their bodies than to limbs, likely as a result of “growing” outwards.

Thank you. We will take a look at this. Currently we are working through a backlog of communication.

Yes. Several other applications (both with and without an underlying 2D grid) are in the pipeline which we hope to publicise soon.

I posted about this and typed in some stuff from his book earlier:

https://news.ycombinator.com/item?id=21858465

John von Neuman's 29 state cellular automata machine is (ironically) a classical decidedly "non von Neumann architecture".

https://en.wikipedia.org/wiki/Von_Neumann_cellular_automaton

He wrote the book on "Theory of Self-Reproducing Automata":

https://archive.org/details/theoryofselfrepr00vonn_0

He designed a 29 state cellular automata architecture to implement a universal constructor that could reproduce itself (which he worked out on paper, amazingly):

https://en.wikipedia.org/wiki/Von_Neumann_universal_construc...

He actually philosophized about three different kinds of universal constructors at different levels of reality:

First, the purely deterministic and relatively harmless mathematical kind referenced above, an idealized abstract 29 state cellular automata, which could reproduce itself with a Universal Constructor, but was quite brittle, synchronous, and intolerant of errors. These have been digitally implemented in the real world on modern computing machinery, and they make great virtual pets, kind of like digital tribbles, but not as cute and fuzzy.

https://github.com/SimHacker/CAM6/blob/master/javascript/CAM...

Second, the physical mechanical and potentially dangerous kind, which is robust and error tolerant enough to work in the real world (given enough resources), and is now a popular theme in sci-fi: the self reproducing robot swarms called "Von Neumann Probes" on the astronomical scale, or "Gray Goo" on the nanotech scale.

https://en.wikipedia.org/wiki/Self-replicating_spacecraft#Vo...

https://grey-goo.fandom.com/wiki/Von_Neumann_probe

>The von Neumann probe, nicknamed the Goo, was a self-replicating nanomass capable of traversing through keyholes, which are wormholes in space. The probe was named after Hungarian-American scientist John von Neumann, who popularized the idea of self-replicating machines.

Third, the probabilistic quantum mechanical kind, which could mutate and model evolutionary processes, and rip holes in the space-time continuum, which he unfortunately (or fortunately, the the sake of humanity) didn't have time to fully explore before his tragic death.

p. 99 of "Theory of Self-Reproducing Automata":

>Von Neumann had been interested in the applications of probability theory throughout his career; his work on the foundations of quantum mechanics and his theory of games are examples. When he became interested in automata, it was natural for him to apply probability theory here also. The Third Lecture of Part I of the present work is devoted to this subject. His "Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components" is the first work on probabilistic automata, that is, automata in which the transitions between states are probabilistic rather than deterministic. Whenever he discussed self-reproduction, he mentioned mutations, which are random changes of elements (cf. p. 86 above and Sec. 1.7.4.2 below). In Section 1.1.2.1 above and Section 1.8 below he posed the problems of modeling evolutionary processes in the framework of automata theory, of quantizing natural selection, and of explaining how highly efficient, complex, powerful automata can evolve from inefficient, simple, weak automata. A complete solution to these problems would give us a probabilistic model of self-reproduction and evolution. [9]

[9] For some related work, see J. H. Holland, "Outline for a Logical Theory of Adaptive Systems", and "Concerning Efficient Adaptive Systems".

https://www.deepdyve.com/lp/association-for-computing-machin...

https://deepblue.lib.umich.edu/bitstream/handle/2027.42/5578...

https://www.worldscientific.com/worldscibooks/10.1142/10841

Ackley is literally thinking hundreds of years ahead of the hardware we have. It's a shame that his ideas will probably never make much of a dent in the current computing ecosystem

This is the foundation for 'programming' morphogenesis of multi-cellular synthetic life in the far future. Of course you first have to get the programming of monocelluar life to work.

Also, the reverse might be possible, i.e., decompiling genetic code from phenotype and so on.