Instead of characters -> 'byte-pair-encoding'-like sequences -> words -> sentences, think primordial peptides -> simple protein parts -> more complicated protein components -> proteins. If this "protein linguistic hypothesis" is correct, I see no reason why the manifold wouldn't be discoverable and learnable with modern SGD techniques.
If we accept his comparison to other results it seems RGNs have an unreasonable effectiveness for topologies...
But for sure, there is also a boundary layer, for interactions between cells. This would have to represent an almost entirely different set of chemical interaction rules for signaling, with its own constraints, minimum requirements, and optional expressions.
So, it's useful to conceptualize in terms like this, but problems solved within the context of intracellular operations will only offer clues about tissue organization, and indeed, tissue requirements may drive the optional intracellular interactions more often than not, rather than the reverse. In cases where intracellular interactions drive extracellular organization, it's essentially leaky abstractions dictating the details of higher level implementation.
This seems reasonable, but another possibility is that modern proteins (domains) are 'carved' from larger proteins that had a looser structure.
In other words, primordial proteins could have been badly folded and mutations gradually improved them to smaller, better folded structures.