https://news.ycombinator.com/item?id=43625474 "Obituary for Cyc"
https://news.ycombinator.com/item?id=40069298 "Cyc: History's Forgotten AI Project"
It's a bit odd that their paper doesn't even mention CYC once.
[1] I could be wrong, but even those that mention Cyc use it only as a historical example of early work in KRR / symbolic AI. [2] OpenCyc being the small subset which is available, tho I haven't met anyone who worked with it.
{
"causal_relation": {
"cause": {
"concept": "boom"
},
"effect": {
"concept": "bust"
}
}
}
An inventory of casual knowledge would be really fun, although it's hard to think what it would consist of now that I think about it...
There is this concept of "hidden knowledge" about all the things you know at work that no one really thinks about is knowledge so it's hard to let newcomers know about it.
But that does sound different than "casual knowledge", and so does "trivia".
Oh well!
A time traveller's wiki that collects casual knowledge for different times (and different places).
Such as: "Buying a train ticket in Paris in 1972".
But it was a shower thought and it's pretty hard to imagine how this knowledge should be collected and especially presented.
In a way, wikipedia is already doing this by keeping records of articles as they change over the years :)
The article about train tickets wasn't so good as an example but "computer monitor" from 2004 is kind of fun to read :)
Unfortunately, "casual knowledge" is often omitted when writing informative articles. In this example, there is no mention that power buttons are often located somewhere in the back of the monitor, which was good to know in 2004. Also, some monitors are drawing power from the computer, thus they won't power up before the computer will. And speaking of that: You may want to turn of your computer after shutdown!
Edit: This would probably be useful for novelists and filmmakers (in addition to the casual time traveller)
But I also wonder if that is actually an equivalent.
I think of "common sense" as think you should know to function well in your current society.
I really don't know what "casual knowledge" would mean. In my head it's some kind of low stakes knowledge for everyday life (but more 'useful' than trivia).
Maybe the order is "common sense", "casual knowledge" and trivia?
Even more importantly, the endpoints of each such causative arrow are also complex, fuzzy things, and are best represented as vectors. I.e.: diseases aren't just simple labels like "Influenza". There's thousands of ever-changing variants of just the Flu out there!
A proper representation of a "disease" would be a vector also, which would likely have interesting correlations with the specific genome of the causative agent. [1]
Next thing is that you want to consider the "vector product" between the disease and the thing it infected to cater for susceptibility, previous immunity, etc...
A hop, skip, and a small step and you have... Transformers, as seen in large language models. This is why they work so well, because they encode the complex nuances of reality in a high-dimensional probabilistic causal framework that they can use to process information, answer questions, etc...
Trying to manually encode a modern LLM's embeddings and weights (about a terabyte!) is futile beyond belief. But that's what it would take to make a useful "classical logic" model that could have practical applications.
Notably, expert systems, which use this kind of approach were worked on for decades and were almost total failures in the wider market because they were mostly useless.
[1] Not all diseases are caused by biological agents! That's a whole other rabbit hole to go down.
One quibble, and really mean only one:
> a high-dimensional probabilistic causal framework
Deep learning models aka neural network type models, are not probabilistic frameworks. While we can measure on the outside a probability of correct answers across the whole training set, or any data set, there is no probabilistic model.
Like a Pachinko game, you can measure statistics about it, but the game itself is topological. As you point out very clearly, these models perform topological transforms, not probabilistic estimations.
This becomes clear when you test them with different subsets of data. It quickly becomes apparent that the probabilities of the training set are only that. Probabilities of the exact training set only. There is no probabilistic carry over to any subset, or for generalization to any new values.
They are estimators, approximators, function/relationship fitters, etc. In contrast to symbolic, hard numerical or logical models. But they are not probabilistic models.
Even when trained to minimize a probabilistic performance function, their internal need to represent things topologically creates a profoundly "opinionated" form of solution, as apposed to being unbiased with respect to the probability measure. The measure never gets internalized.
Not so sure one should take stories about who said something in ancient times at face value ;)
Virgil.
[0] https://en.m.wikipedia.org/wiki/Felix,_qui_potuit_rerum_cogn...
What's perhaps different is that the machine, via LLM's, can also have an 'opinion' on meaning or correctness.
Going fully circle I wonder what would happen if you got LLM's to define the ontology....
https://deepsense.ai/resource/ontology-driven-knowledge-grap...
>hammering out an ontology for a particular area just results in a common understanding between those who wrote the ontology and a large gulf between them and the people they want to use it
This is the other side of the bitter lesson, which is just the empirical observation of a phenomenon that was to be expected from first principles (algorithmic information theory): a program of minimal length must get longer if the reality it models becomes more complex.
For ontologists, the complexity of the task increases as the generality is maintained while model precision is increased (top down approach), or conversely, when precision is maintained the "glue" one must add to build up a bigger and bigger whole while keeping it coherent becomes more and more complex (bottom up approach).
States like "human_activity" are not objectively measurable.
Fairly PGMs and causal models are not the same, but this way of thinking about state variables is an incredible good filter.
Well, or at least they would need a heavy dose of operationalisation.
Which is directly usable knowledge if you are building out a causal graph.
In the meantime, a cause and effect representation isn't limited to only listing one possible effect. A list of alternate disjoint effects, linked to a cause, is also directly usable.
Just as an effect may be linked to different causes. Which if you only know the effect, in a given situation, and are trying to identify cause, is the same problem in reverse time.
We have quite a good understanding that a system cannot be both sound a complete, regardless people went straight in to make a single model of the world.
Huh, what do you mean by this? There are many sound and complete systems – propositional logic, first-order logic, Presburger arithmetic, the list goes on. These are the basic properties you want from a logical or typing system. (Though, of course, you may compromise if you have other priorities.)
Honestly, I don’t know understand how these so-ontologies have persisted. Who is investing in this space, and why?
So, by design, it's pretty useless for finding new, true causes. But maybe it's useful for something else, such as teaching a model what a causal claim is in a deeper sense? Or mapping out causal claims which are related somehow? Or conflicting? Either way, it's about humans, not about ontological truth.
A coronavirus isn't "claimed" to cause SARS. Rather, SARS is a name given to the disease cause by a certain coronavirus. Or alternatively, the name SARS-nCov-1 is the name given to the virus which causes SARS. Whichever way you want to see it.
For a more obvious example, saying "influenza virus causes influenza" is a tautology, not a causal relationship. If influenza virus doesn't cause influenza disease, then there is no such thing as an influenza virus.
> Ontologies and all that have been tried and have always been found to be too brittle.
I'd invite you to look at ontologies as nothing more than representations of things we know in some text-based format. If you've ever written an if statement, used OOP, trained a decision tree, or sketched an ER diagram, you've also represented known things in a particular text-based format.
We probably can agree that all these things are ubiquitous and provide value. It's just that those representations are not serialized as OWL/RDF, claim less about being accurate models of real-world things, and are often coupled with other things (i.e., functions).
This may seem reductionist in the sense of "we're all made of atoms", but I think it's important to understand why ontologies as a concept stick: they provide atomic components for expressing any knowledge in a dedicated place, and reasoning about it. Maybe the serializations, engines, results or creators suck, or maybe codebase + database is enough for most needs, but it's hard to not see the value of having some deterministic knowledge about a domain.
If you take _ontology_ to mean OWL/RDF, this paper wouldn't qualify, so I'm assuming you took the broader meaning (i.e., _semantic triples_).
> Take the examples from the front page (which I expect to be among the best in their set)
Most scientific work will be in-progress, not WordNet-level (which also needs a lot of funding to get there). You ideally want to show a very simple example, and then provide representative examples that signal the level of quality that other contributors/scientists can expect.
Here, they're explicit about creating triples of whatever causal statements they found on Wikipedia. I wouldn't expect it to be immediately useful to me, unless I dedicate time to prune and iron out things of interest.
> human_activity => climate change. Those are such a broad concepts that it's practically useless.
Disagree. If you had one metric that aggregated different measurements of climate change-inducing human activity, and one metric that did the same for climate change, you could create some predictions about N-order effects from climate change. Statistical analysis anyway requires you to make an assumption about the causal relationship behind what you're investigating.
So, if this the level of detail you need, this helps you potentially find new hypotheses just based on Nth order causal relations in Wikipedia text. It's also valuable to show where there is not enough detail.
> Or disease => death. There's no nuance at all.
Aside from my point above - haven't looked at the source data, but I doubt it stops at that level. But even if it does, it's 11 million things with provenance you can play with or add detail to.
Or you can also show that your method or choice of source data gets more conceptual/causal detail out of Wikipedia, or that their approach isn't replicable, or that they did a bad job, etc. These are all very useful contributions.
That's because we know how to interpret the concepts used in these representations, in relation to each other. It's just a syntactic change.
You might have a point if it's used as a kind of search engine: "show me wikipedia articles where X causes Y?" (although there is at least one source besides wikipedia, but you get my drift).
> Aside from my point above - haven't looked at the source data, but I doubt it stops at that level.
It does. It isn't even a triple, it's a pair: (cause, effect). There's no other relation than "causes". And if I skimmed the article correctly, they just take noun phrases and slap an underscore between the words and call it a concept. There's no meaning attached to the labels.
But the higher-order causations you mention are going to be pretty useless if there's no way on how to interpret them. It'll only work for highly specialized, unambiguous concepts, like myxomatosis (which is akin to encoding knowledge in the labels themselves), and the broad nature of many of the concepts will lead to quickly decaying usefulness when the length of the path increases. Here are some random examples (length 4 and 8, no posterior selection) from their "precision" set (197k pairs):
['mistake', 'deaths', 'riots', 'violence']
['higher_operating_income', 'increase_in_operating_income', 'increase_in_net_income', 'increase']
['mail_delivery', 'delays', 'decline_in_revenue', 'decrease']
['wastewater', 'environmental_problems', 'problems', 'treatment']
['sensor', 'alarm', 'alarm', 'alarm']
['thatch', 'problems', 'cost_overruns', 'project_delays']
['smoking_pot', 'lung_cancer', 'shortness_of_breath', 'conditions']
['older_medications', 'side_effects', 'physical_damage', 'loss']
['less_fat', 'weight_loss', 'death', 'uncertainties']
['diesel_particles', 'cancer', 'damages', 'injuries']
['malfunction_in_the_heating_unit', 'fire', 'fire_damage', 'claims']
['drug-resistant_malaria', 'deaths', 'violence', 'extreme_poverty']
['fairness_in_circumstances', 'stress', 'backache', 'aching_muscles']
['curved_spine', 'back_pain', 'difficulties', 'stress', 'difficulties', 'delay', 'problem', 'serious_complications']
['obama', 'high_gas_prices', 'recession', 'hardship', 'happiness', 'success', 'promotions', 'bonuses']
['financial_devastation', 'bankruptcy', 'stigma', 'homelessness', 'health_problems', 'deaths', 'pain', 'quality_of_life']
['methylmercury', 'neurological_damage', 'seizures', 'changes', 'crisis', 'growth', 'problems', 'birth_defects']
This one is cherry-picked, but I found it to funny to omit:
['agnosticism', 'despair', 'feelings', 'aggression', 'action', 'riot', 'arrest', 'embarrassment', 'problems', 'black_holes']LLMs are such a huge improvement that the only real use I see in projects like Cause et, the defunct OpenCyc project, etc. the only possible practical use might be as a little extra training data.
Currently a lot of people research goes in the direction that there is "data uncertainty" and "measurement uncertainty", or "aleatoric/epistemic" uncertainty.
I foumd this tutorial (but for computer vision ) to be very intuitive and gives a good understanding how to use those concepts in other fields: https://arxiv.org/abs/1703.04977
Pretty bold to use a picture of philosophers as your splash page and then make a casual claim like this. To say the least, this is an impossible task!
The tech looks cool and I'm excited to see how I might be able to work it into my stuff and/or contribute. But I'd encourage the authors to reign in the rhetoric...
Building a perfectly accurate model of the world isn't possible. We need to create tools that make it easier for regular people to build more accurate models, not delude ourselves with dreams of perfection.
"A causes B" usually implies that A and B are positively correlated, i.e. P(A and B) > P(A)×P(B), but even that isn't always the case, namely when there is some common cause which counteracts this correlation.
Thinking about this, it seems that if A causes B, the correlation between A and B is at least stronger than it would have been otherwise.
This counterfactual difference in correlation strength is plausibly the "causal strength" between A and B. Though it doesn't indicate the causal direction, as correlation is symmetric.
On the other hand, what we seem to have with LLM models, and the transformer approach in particular, is a sort of probable statistical correlation, calculated by brute-forcing and approximation (the gradient descent). So this is not true causation also, it becomes one only after a human observes it and agrees it follows certain causality.
/Not sure whether I can state that it is also material but in another non-logical sense, perhaps would sound nonsensical, but the apparent logical structure in the LLM production rather emerges from training patterns, not from explicit logical operations./
There's nothing wrong having a graphical structure which models causality, and of course - this needs to be discovered first. But then we have LZW/Sequitur using very brute-force way in order to find the minimal grammar for compressing certain data lossless-ly, thus discovering some logical structure (and correlation), but this is not yet causation. Indeed finding patterns != finding causal relationships.
My gut feeling is we want something that would result in a correct PROLOG-like set of inference rules, but based on actual causality, not conflating correlation. And then this - for a larger corpus - world's knowledge, but we don't have the means (yet) to figure out the correlation, even though approaches exist for smaller corpus.
It is perhaps the gradient descent and the fact that this composition of tensor algebra is differentiable that is the ingenious thing about the ML we deal with now, but everyone is dreaming of some magic algo which would allow finding the causation so that it results in non-probabilistic graphical model, or at least a model that we can follow the stochastic branching on in a observable manner.
It is indeed ingenious to fold multi-dimensional spaces, multiple times, in order to disambiguate the curvature of bunny's ear from the one of bear's ear. But it just does not feel right to do logic and causation by means of differential calculus and stochastic structures.
Essentially one totalizing meta analysis.
E.g. If it reads an article about the relationship between height and various life outcomes in Indonesian men, then first, it would store the average height of Indonesian men, the relationship between the average height of Indonesian men and each life outcome in Indonesian men, the type of relationship (e.g. Pearson's correlation), the relationship values (r value), etc. It would store the entity, the relationship, the relationship values, and the doi source.
Something like a quantitative Wikipedia.
Alcohol causes anxiety. At the same time it causes relaxation. These effects depend on time frame, and many individual circumstances.
This is a single example but the world is full of them. Codifying causality will involve a certain amount of bias and belief. That does not lead to a better world.
[0] https://en.wikipedia.org/wiki/Causal_graph
Harvard has a free course about it: https://www.edx.org/learn/data-analysis/harvard-university-c...
This is such a cool schemaless approach and has so much potential for open data linking, classical reasoning, LLM reasoning. But open data (together with RSS) has been dead for a while as all big companies have become just data hoarders. And frankly, while the concept and the possibilities are so cool, the graph databases are just not that fast and also not fun to program.
Unfortunately, frequency is the primary way AI works, but it will never be accurate for causality because causality always has the dynamic that things can happen just “because”. It’s hacked into LLMs via deliberate randomness in next-token prediction.
Random thoughts about organizing knowledge:
- Categories need fractal structures.
- Categories need to be available as subcategories to other categories as a pattern.
- Words need to be broken down into base concepts and used as patterns.
- Social information and context alter the meaning of words in many cases, so any semantic web without a control system has limited use as an organization tool.
{
"causal_relation": {
"cause": {
"concept": "vaccines"
},
"effect": {
"concept": "autism"
}
}
},
>> "Even though the article was fraudulent and was retracted, 1 in 4 parents still believe vaccines can cause autism."
>> On 28 February 1998 Horton published a controversial paper by Dr. Andrew Wakefield and 12 co-authors with the title "Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children" suggesting that vaccines could cause autism.
>> He was opposed by vaccine critics, many of whom believe vaccines cause autism, a belief that has been rejected by major medical journals and professional societies.
All that I've seen don't actually say that vaccines cause autism
"vaccines > autism"
because
"Even though the article was fraudulent and was retracted, 1 in 4 parents still believe vaccines can cause autism."
I think this could be solved much better by using even a modestly powerful LLM to do the causal extraction... The website claims "an estimated extraction precision of 83% " but I doubt this is an even remotely sensible estimate.