undefined | Better HN

0 pointskibwen6y ago0 comments

> I think the actual pain in Python 2 came from the misguided decision not to adopt UTF-8 as the default character encoding

The decision of a default encoding surely dates back to Python 1.0 or earlier, which predates not just UTF-8 but even Unicode itself. Python is an old language!

And if the assertion is that Python 2.0 should have made the tumultuous Unicode jump when it released in 2000, I could get behind that (especially in retrospect!), but enthusiasm for both Unicode and UTF-8 was not nearly as high then as it is today, so I don't begrudge them for not jumping at the opportunity.

0 comments

3 comments · 2 top-level

steveklabnik6y ago· 1 in thread

Interestingly enough, Ruby 1.8 -> 1.9, the big version jump there, there was this kind of transition. The remainder of this post is all IIRC, it's been a while...

Ruby 1.8 had "everything as bytes" and there was no concept of encodings.

Ruby 1.9 introduced explicit encodings on every string. By default, strings would be encoded as the same encoding as your source file. The default was ASCII. You could control this explicitly with a magic comment, and so many folks added the "UTF-8" comment, to get strings encoded as utf-8 by default.

Ruby 2.0, which was not as large a transition as Ruby 1.8 -> 1.9, even though it sounds like a larger one, said that encodings of files were UTF-8 by default, and therefore, strings generally became UTF-8 by default as well. Most folks just removed their magic comments.

mark-r6y ago

It's surprising how many people believe that you can use a magic comment to make Python use UTF-8 encoding as the default. All the magic comment affects is the encoding of the source file, not the run-time.

morelisp6y ago

Enforcing UTF-8 as the default encoding, barring a magic comment otherwise, would hardly have been the biggest compatibility break in the 2.x line. It could have been done in any minor release, IMO.

j / k navigate · click thread line to collapse