Hence, these people moved on the using Category Theory, which may or may not lead to the use of GNNs.
Reading the first part of the present paper, the Topological DL would instead move beyond the idea of "pairwise relation".
That is also interesting me regarding Geometric Deep Learning, which got some hype and interest recently, and seemed like a good start for more formal representation of different deep learning models (finding connections and mathematical steps between the model zoo). Something more mathematically rigorous does seem needed to truly make informed engineering improvements and scientific understanding.
Maybe someone else has the knowledge to give you some more precise answers.
As far as I understand we need to differentiate between higher order relations that can be learned by message parsing and those that cannot. Some higher order relationships can be represented in a graph (of pairwise relations) and, furthermore, can be learned by iterating out from a focal node through the paths (think centrality). GNNs can learn those.
But GNNs can't learn all Graphs because not all such relations are amendable to message parsing. So, not all higher order information contained in a graph are learned by GNNs.
In the present case, the authors seem to make the point that there are higher-order relations (say, node-set) which are also not representable by a graph (or a collection of graphs).
I actually do not know whether that is obviously true. For instance, hypergraphs can be modeled as a collection of graphs. But if the claim stands, then we'd need topological deep learning to move further. Otherwise, like you say, it is maybe a more general framework to understand information processing in DL - or more efficient to learn node-node or node-set information.
Either way, I have not yet quite figured out how Category Theory and Topological DL relate to each other (each generalising Geometric Deep Learning).
- https://petar-v.com/ - Petar's homepage
- https://scholar.google.co.uk/citations?user=kcTK_FAAAAAJ&hl=... - Petar's Google Scholar
- https://www.linkedin.com/in/petarvelickovic/ - Petar's LI
It definitely feels like graphs/topology should be helpful tools to work with data(Since graph-like structures are good representations of the real world), but we need to solve this efficiency issue before this can be possible.
Also to address the confusion on how category theory comes into it, category theory studies abstract structures where you have objects and relationships between these objects. A lot of algebraic topology(Which is the sort of topology relevant here) is built in the language of category theory(Either by neccesity or by convention).
