Grammars of Formal Uncertainty (opens in new tab)

This seems like the type of work that, whether or not it itself is profound (which I believe it seems it is), will accelerate the forthbringing of derivative works that will definitely be profound.

I do not see much backing to the claim LLMs can "democratize" formal methods, given that their specifications likely still have to be proof-read by an actual expert. There's also the issue that plain assertion-checking will only take you so far: formal specifications typically need to account for the passing of time, and then the tailoring of the verification platform to the system, rather than property specification, is what usually takes the most effort.

But the general approach to uncertainty and connections to OoD detection do sound interesting.

Great breakdown—PCFG-based uncertainty metrics seem like exactly what we need to make LLM-SMT pipelines robust and reliable!

Brings out a refreshing perspective to LLM guarantees. Very good work.