This idea stemmed from long-running frustration every time I hand-wrote JSON (particularly common for Node/npm packages and test cases). After stewing on it for over a year, I bit the bullet today and made it a reality.
Feedback greatly welcome!
1) to be language agnostic. Following ES5's syntax directions is fine for ES5 focused development, possibly not for others.
2) to be final - i.e. there is no version number. If you want to extend or build something else, I am fairly certain Douglas would prefer that you call it something else. See http://inkdroid.org/journal/2012/04/30/lessons-of-json/ at 5:30 in the video.
Don't take these points as criticisms, I too would like to be able to use single quotes and trailing commas and everything else here. I think though that in order to work with JSON as a static spec, an alternative approach could be to have your project be a layer like HAML, SASS or CoffeeScript. Each of those tools compiles or transforms to conforming input for their target specs rather than replacing them.
I'm aware that JSON is final — it's my hope that a new format that's easier to write (whether this or something else) picks up steam — but thanks for the feedback on the name! I consciously used a different file extension (.json5) to avoid conflicts; hopefully that's a good start.
I already use (and love) CoffeeScript, but my motivation for building this was to have tools use it natively, so that sibling files in a different format wouldn't have to be maintained alongside the needed JSON.
The support for evaling in the various engines doesn't matter because you should not be evaling JSON strings for security purposes.
The serialization format is strict for a reason: look at where less strict HTML parsing has gotten us.
I also love to make tests data-driven where possible. JSON is the natural format for data on Node, and again, test cases are usually written by hand (before code exists to generate or validate them).
I'm not sure I understand your point about eval'ing. This parser doesn't use eval(), and it works in every modern browser.
Finally, I'm not advocating for arbitrary looseness. These additions conform directly to ES5's additions.
I think this fixes most of the problems of using it in hand-written situations. Just the lack of comments in JSON makes it inappropriate for configuration files and tests.
But this shouldn't be called JSON. JSON is strict and doesn't include comments because it's designed for interoperability between different systems. Just because it's been abused to be used for configuration files doesn't mean it needs fixing. One just needs to use the right tool for the job. Your project might be the right tool but then it's not JSON.
If I had a penny for every time a trailing comma in a json config file bit me, I would have...many pennies!
I am indifferent about single vs double quotes. The backslash newline escape seems 'messy'.
Not a fan of keys not being quoted. That stikes me as wrongish for some reason that I can't put my finger on.
This "JSON5" thing is an abomination. Use whatever quotes you like! More string escaping rules! Adding these things would make JSON worse, not better.
Here's another idea: why not write a tool that tidies up your hand-written JSON files? You could even re-use your "JSON5" parser to do it. That way you can be as lazy as you like when writing files, but not expect the rest of the world to learn your new dialect.
My motivation for this was actually hoping that this gets adopted in some environments that use JSON. Node relies on package.json files, for example, which frequently need to be hand-edited. Maintaining a sibling file in a different format feels equally wrong to me as it seems JSON5 seems to you. =)
Two or three "improvements" like quotes, comments don't legitimate a new standard. BTW, you could also leave out the colons.
As for your question: of course it's possible. Write a tool that translates between your preferred format and JSON. Set up your development environment to automatically convert between the two formats when editing JSON files.
The syntax is not necessarily nicer, it has only a few changes in the lexical structure and the syntactic structure is pretty much the same. If it's a little nicer for a human writing it, it's a little hairier for a parser to parse it.
But how much you spend time writing JSON anyway? Most of the time JSON is read and written by tools. Human intervention is rare.
In addition to that, the native JSON parsers in the browsers are probably implemented in native code, and this parser of yours is written in JavaScript. So it wouldn't be a surprise if this was an order of magnitude slower than a native JSON parser.
The package.json files, by and large, don't have a multiline string problem. For trailing and missing commas, I think you could make something that automatically fixes them, if that's the only problem, and run it before npm is run, perhaps using a shell alias.
JSON's strength and it's very reason for existing is its simplicity and unambiguity.
I'm not sure why you lump quoting keys with semicolons, though; his regular JS style guide doesn't discourage quoting object key (neither does JSLint).
Oh, he was never elected? Why the fuck should I care about what one man thinks then?
IMHO, aggressive comments that contain nothing but sarcasm are not welcome here. Let's try and keep the place a bit friendly (or at least grown-up and professional), even if we disagree. If you are going to be sarcastic, at least have the decency to make an interesting point at the same time.
> When was Crockford elected to set the rules for programmers and why wasn't I given a vote?
Crockford was obviously not "elected" in any official sense. However, he produced something that the programming language needed, and it very quickly became a standard. Have you ever used JSON in a project? Then you implicitly voted for JSON and, by extension, Crockford.
> Oh, he was never elected? Why the fuck should I care about what one man thinks then?
Because the rest of the industry has decided that JSON's benefits outweigh its downsides, and using it is a net positive. JSON is not perfect, but it solves an annoying problem rather elegantly.
If you've never read his book or watched any of his talks, maybe you should check them out. He never says "this is how you have to do it". He says something to the effect of 1) JavaScript is a flawed language. 2) DC has used/helped develop it for a long time 3) he has figured out a lot of least bad solutions and habits to try and for to mitigate common problems. If you don't want to follow his suggestions fine, but don't be surprised when you run into the problems that he created them to help you avoid.
Any changes to JSON should be fought til death. Use YAML or make a new format if you need more.
FWIW, I'm not adding any new data types or functionality, just making the syntax more human-friendly. It also continues to be a strict subset of ECMAScript, but v5 now.
I see where you were going with it, ES5 support as a baseline is nice, also comments are nice but also bulky to a streamlined messaging format in most cases (config is usually yaml or some other format for this reason maybe).
I guess the naming is what maybe might cause some confusion and a good engineer makes things more simple not more complex, the simplicity of JSON is hard to achieve with such success but it is also its killer feature so that should be recognized. As long as we don't have to support 50 different 'modern JSON' formats then we are good. There are some good ones like BSON for binary JSON. I guess JSON5 works for your ES5 support.
I think here you have good reasons for your work but it goes back to the age old be 'conservative in what you send, liberal in what you accept'. That standards agreement makes everything work and to me being an engineer is taking complexity and simplifying always, bloat is bad. Here it isn't that but XML started out simple as well and became a verbose, bloated hell so people get touchy with JSON hehe.
Future newbie developers will wonder why their xml reader has problems reading document.xml2 without actually realizing the format is different from xml.
It's worse in your case since JSON5 almost looks like regular JSON, but if one were to use one of your special features, it would cause any normal JSON parser to break. The newbie developer might not be sharp enough to realize he needs a completely different parser.
* Comments in JSON files are not a great idea because people might (will?) abuse them to add special instructions for their own serializers. As soon as this happens we'll have a format that is not compatible with other serializers.
* JSON today is a universal format. So, what are special characters? Should '1hello' be quoted? If no, then its not a subset of JavaScript anymore. If you're writing a new specification you have to define all those things. Same with multiline strings: space after backslash?
* Why? I know that hand-writing JSON can be annoying but is this really the biggest problem we have?
Special characters: sorry I didn't write a formal spec =), but the intent is to be a pure subset of ES5 just like regular JSON is a pure subset of ES3. Unquoted object keys in ES5 can contain only letters/numbers/_/$, and only begin with letters/_/$. So yes, '1hello' would need to be quoted, just like regular ES5.
I'm not sure how to respond to your why other than what I've already written. I never claimed this was the biggest problem we have. =)
Anyway, I wouldn't focus on JavaScript engines because JSON is now a universal format so for languages other than JavaScript you will have to define a formal specification.
Opening paragraph: "JSON is strict. Keys need to be quoted; strings can only be double-quoted; objects and arrays can't have trailing commas; and comments aren't allowed."
If this represents a comprehensive list of the problems JSON5 solves I think you'll have a hard time rallying people around your cause.
I find JSON tedious and error-prone to generate by hand. I also frequently wish I could document the data with comments.
Others have written about these same ideas, e.g. [1]; after feeling the frustration for over a year, I decided it might be productive to actually build a JS parser to get the conversation started.
My main use cases are Node/npm package.json files, and test case data.
I whole-heartedly agree with you on the tediousness, but JSON has this dubious double-nature that it is sometimes written by hand but mostly used as a protocol. As a protocol we want (and I would even argue need) this strictness since it makes day-to-day operations easier and parsers generally more reliable (how many times have we seen someone implement their own JSON parser? Now, what about XML? The first is a nice evening the latter is, well, not...).
As for package.json, I would argue the problem is their choice of serialisation rather than with JSON itself. Right tool for the wrong job. YAML would probably be a better pick.
Why do you need to generate it by hand? Create object structures in whatever language you are working in and then simply serialize them to JSON.
Yes, JSON is a pain to edit/write by hand at times, but that's an editing problem and therefore should be solved in the editor. In any halfway decent editor it should be trivial to extend it with helpers for editing JSON. Perhaps have it convert .json files to YAML(etc) on opening, and back to .json on save etc.
This is a problem that plagues many technical people (present company included). You had a good instinct, but then took a left turn somewhere. "JSON is hard to hand edit...let's undo all the speed/compatibility progress we've made with JSON in the last ~8 years and introduce flaws in the protocol with a new non-standardized format...all so it's superficially easier to edit by hand."
You're definitely right that some of the hand-editing problems could in theory be solved by tools, but it unfortunately doesn't address documentation/comments.
I've heard the chorus on YAML though, and I'll definitely be looking into it. Thanks for the feedback!
If we're going to undo years of progress we should do it for a very good reason.
JSON has wide support in many languages. Some of them have fast low level parsers. All that work has to be duplicated. Again, this just doesn't seem sufficient reason.
Right now if I want to throw together a Python script to crawl a bunch of NPM packages and read their package.json (let's say to put together a chart or collect statistics), I can easily.
If NPM switches to JSON5, we lose compatibility without a lot of duplicated effort. Again, there's just not enough benefits.
I consider comments a flaw, because they add little benefit. JSON is a data serialization/interchange format and there shouldn't be much need to comment on it, it should be largely self explanatory. If you really need to tag things with info, you can in JSON. Adding comments is to tempting for people to abuse.
However, if you really need lots of comments, there are many existing formats which might be better suited for your task. JSON is ultimately for machines. It has a benefit that humans can read/edit it easily...but if you have data that is intended more for humans and needs lots of meta information like a complicated config file, you should either create a front end for it or use a format better suited to that purpose.
When you consider that writing a plugin for your editor, or just a stand-alone translator would solve all of the problems this does, there's no reason to put up with all the drawbacks.
What it comes down to IMHO, is this isn't suitable to replace JSON and for other uses there are already existing formats that are better.
The justification for not having comments in JSON is that in the great disaster that was XML, some projects would parse the comments and take them as semantically significant. However, the real problem there was that parser libraries would expose the comments, and that some generators would put important information only in comments. But I think that these mistakes are unlikely to be repeated, and that the proposed alternative - moving all comments into the markup, or eliminating them entirely - is just obviously worse.
One of the things that ruined the XML ecosystem was a persistent belief that XML was to be read and written by machines, combined with a reality in which it was mostly used for human-written config files, leading to a tolerance for awful syntax (like prefixing every single attribute with a namespace, and the ridiculous CDATA notation). I'm seeing the same thing with JSON: A significant fraction of its use is for human-written and human-read config files (which often desperately need comments) and people are pretending it's strictly a data interchange format that shouldn't be used for that.
It is sometimes said that JSON was discovered, rather than invented - that the syntax was already out there. So it is with JSON5: There is nothing new in this, it is simply return to JavaScript Object Notation and bringing in the rest of what Javascript has.
So, all you pooh-poohing ideologists: please seriously rethink whether disallowing comments, trailing commas, and quoteless member names is actually a good idea. Consider this in light of the fact that JSON is widely used, today, as a config-file language, and that {"--":"This is a comment"} is ridiculous enough that no one does it in practice, can't be inserted on any line, and invites consumers of your data to parse the comments.
(As a friend pointed out, "Do you not comment your code just because it's meant for machines?")
People here gave you enough reasons.
Who actually uses octal, except by accident?
If you really want ES5 for writing package.json files, then you could save the time and effort by just using ES5:
var _ =
{
some: 'object literal',
// a comment
num: 3,
}
fs.writeFileSync(path.basename(__filename, '.js') + '.json', JSON.stringify(_))As others have noted this is not JSON and is not necessarily an improvement to JSON - so calling it JSON5 is presumptuous.
Embrace the criticism. Instead of JSON5 call it NotJSON - a standard that breaks the JSON rules for a few narrow use cases - namely when you want a hand-written, commented JSON-like syntax.
Suggestion: turn these criticisms to your favor. Make this into something that promotes good form. Perhaps by acting as a compiler that transforms your forgiving JSON into valid JSON. Make a command line utility. Emphasize how the output passes JSONlint or something. In other words, I'm suggesting that you remake this into a sort of coffeescript for JSON.
In any case, the path you are on isn't gonna garner much support. Change up your stride a bit and you could be on to something rather nice, though.
I also tried something similar a few years back when I was sick of dealing with XML and coming to terms with the required quotes.
While it's certainly prettier to have native literals and optional quotes, it's simply not worth giving up the interoperability with JavaScript.
JSON has its flaws, but is a surprisingly robust and elegant solution to many data interchange problems. But it is not (and I don't think it was ever intended to be) a language for human-edited configuration and data.
Lets write a python parser who accepts semicolons instead of indentation, just to fix when I forget that i am not writing Javascript; oh, and and make it accept the "function" keyword instead of def.