Ask HN: Why is parsing still inconvenient in 2023?

Most of the time, parsing isn't the problem you want to be solving.

People use JSON because they need to send information over a wire, and JSON serializers are abundant. I agree that this is problematic for a bunch of different reasons:

[1] https://taylor.town/json-considered-harmful

But note that this problem has been solved many, many times:

[2] https://en.wikipedia.org/wiki/Comparison_of_data-serializati...

Formats like MessagePack and Cap'n_Proto have a lot of nice properties.

Writing parsers is not easy. And it's especially not easy when you have a custom format that different people want to do different things with.

---

Btw, I've tried out pretty much every parsing library in Rust, Typescript, and Haskell.

Elm's parsing library is the only one I enjoy using:

[3] https://package.elm-lang.org/packages/elm/parser/latest

I think nearley.js has a cool interface but poor execution:

[4] https://nearley.js.org

I don't believe in the premise of your idea. Most of the time stuff can be done using standard data structures—things like numbers, strings, lists and maps of those and so on. Intermediate structures like JSON are only needed for storage and transmission. Inasfar as the need for storage and transmission in turn necessitates the same recurrent task of data serialization, it totally makes sense to reuse tools that have been around for a long time and been optimized for most use cases—so why re-invent databases, JSON parsers and entire wire protocols when more or less optimized tools of the sort are already there? Google wrote protobufs as a replacement for JSON, and while it does offer things beyond JSON like data typing, the appraisal is not unanimous. It is also hard to replace a mature DBMS like Postgres or SQLite with a from-scratch, purpose-made solution without failing on every single useful metric like throughput, feature completeness and reliability.

Personally I too think parsing should be easier in this day and age; I believe Raku (Perl 6) has made meaningful strides in that direction. Other than that, I feel parsing is somewhat over- and lexing is somewhat underrated, if anything. In my experience lexing is really the step you want most to get data out of a byte sequence, and I agree that that should and could be much easier. FWIW JavaScript's RegExes recently obtained the 'sticky flag' which is ultra-beneficial for lexing. Not sure why that bit took so long.

Parser generators have been around for a very long time...i think the difficulty is in defining the format, at which point it is easier to go the standard way