Watch out: ɢoogle.com isn’t the same as Google.com (opens in new tab)

> Unicode maybe should have been three dimensional, with "concept of G" in the 2D space, and "ways of representing G" behind G, along the third axis. All ways of representing G, whether little capital, capital, lower case, would or at least could equate to conceptual G in the 2D space.

It brings up interesting, long-standing problems. Which of these count as the same letters?

* Letters in two languages with the same appearance and making the same phonetic sound

* Letters in two languages with the same appearance but making slightly different phonetic sounds. E.g., R in English and French

* Letters in in two languages that are otherwise the same, but one has an accent. Is the accent part of the letter? Separate? Are they really the same letter?

* Letters in two languages with the same appearance but making completely different phonetic sounds.

* Similar (by any property) letters in two related languages; e.g., both Indo-European

* Similar (by any property) letters in two unrelated languages; e.g., French and Vietnamese.

* Letters with the same phonetic sound but different appearances.

* Letters with the same appearance, one is phonetic and one an ideograph

* Letters that are otherwise identical, but alphabetize differently in their respective languages

* EDIT: Forgot a key one; Letters that are otherwise identical, but follow different rules of how they combine with the letters around them (a common issue, though not familiar to English speakers).

* Letters that are in all ways identical but belong in different languages. In which languages code group does the letter belong? One? Both? What if the subset of Unicode supported by an application includes one language but not the other?

etc. etc.

That character is from the phonetic alphabet so it's not the "concept of G", it's the concept of a "voiced uvular stop", which happens to looks visually like G. So what Unicode is doing is separating two conceptually different ideas, exactly as intended.

The cases where Unicode has taken similar looking characters and combined them into one have not been successful, Han Unification for example was widely viewed as a misstep and has caused many problems, such as making it impossible to embed certain Japanese characters in Chinese text without higher-level markup.

It actually does do something along those lines, with the "canonical" and "compatible" equivalence rules:

https://en.wikipedia.org/wiki/Unicode_equivalence

As mentioned by others on this thread, the real issue is not with Unicode per se, but rather with the ways that web browsers handle it (or fail to handle it, as the case may be).

That's a really interesting proposal, but I'm afraid it would be difficult to implement in practice. If this third dimension were actually encoded into the number that represents each character, you'd end up with a lot of wasted bits (since most characters probably wouldn't even need the 3rd dimension, or at least as much of it as the heaviest users.) Another option would be to supplement the metadata that already accompanies Unicode characters (which block it is in, the name of the character/block, etc...) This could be done in practice now, but the information would almost certainly just be ignored if it needed to be looked up in a supplemental table. Furthermore, it's difficult to agree on just about anything in Unicode, and classifying all the characters based on concept seems like a Herculean task for a slow-moving body.

Any ideas for how to accomplish this in practice?

The G is part of a block called "IPA extensions." Most of its content is more obviously specialized. This G is a phonetic G.

It's not necessarily the case that any given symbol has a bunch of different Unicode representations; unfortunately G has at least two, though.

https://en.m.wikipedia.org/wiki/IPA_Extensions

http://www.fileformat.info/info/unicode/block/ipa_extensions...

Most browsers should forcibly transcribe this to Punycode[1]:

    https://www.𝙿𝙰𝚈𝙿𝙰𝙻.com/

And yet when I paste this into the latest Firefox it redirects to https://www.paypal.com/

No 301 redirects or anything, the browser just treats it like ASCII, which it is clearly not, it actually happens to be Fullwidth:

https://en.wikipedia.org/wiki/Fullwidth_form

Serious phishing opportunity if you ask me!

[1] https://en.wikipedia.org/wiki/Punycode

That looks a lot like the sort of trademark use that authorities have deemed infringing, I'd expect the registrar to "recover" that unless you've got a clear explanation (like Goolge is your name, even then ... (remember Nissan, Mike Rowe, etc.)).

I registered http://www.4ppl3.com a while back. No potential for abuse really, but I just thought it was fun to have a l33t-speak version of the domain name of one of the world's most litigious companies.

That said, I haven't done anything with it, and I'm not a domain squatter, so if anyone wants it I can hook you up!

This would be a great solution. Allowing unicode characters in domain names is just inviting trouble. I understand that people with non-Latin scripts want domain names in their own language and alphabet, but there are way too many unicode characters that will confuse people about legitimate-looking domain names.

Showing non-ascii in red would be an easy solution for everybody.

"Neat, Sparkasse now even has a colored domain name! Now where did I put my TAN device again?"

- Average User

Cool, my head of marketing department wants a RED domain.

Chinese character domains would be shown in red letters. I think it's a good choice of color. :)

No reason for me to try to pick out specific characters. I won't notice. Plus, it won't work for zero width characters, and I might miss it for really tiny ones.

Give me a popup warning explaining the problem when I try to visit the site, same as I get for certificate problems.

They should already only show punycode for characters inside your locale.

'ɢ' is obviously an exception since (I imagine) it's considered to be in your locale, but maybe it shouldn't be.

It should display each code page with a different color. That would make the schema useful for non-english speaking people too.

It would need a more complex solution than that. For example, this is the website of a local bus company where I live:

http://åbus.fi

The characters are from the Latin character set, but non-ASCII. Highlighting the Å in red would look pretty confusing. And in many countries you want the entire domain name written in non-ASCII characters, depending on the language. E.g. websites in Russia, China, India, etc...

And on by default

the visiblend screenshot on your projects page is dead because of an unresolveable dns href.

the screenshot on your kmap repo[1] was dead as well, until i actually opened it. i'm guessing the jpg isnt generated until somebody clicks on it.

enough cyberstalking for me this evening :p

[1] https://github.com/cjrd/kmap

Awesome site, by the way. I'm just checking out your tutorial on LDA.

Most registries did a better job on constructing their IDN tables than Verisign did. :-(

The most important reason is semantics. If "O" and "0" look alike in a certain font, should we use the same character code for both? No, because they have different meaning.

Here are some contexts in which this semantic difference is important: search (compare search results for "cop" and "сор"), alphabetical sorting, text-to-speech, spellchecking, case conversion ("ATOM" -> "atom", but "АТОМ" -> "атом", note the difference between t-т and m-м).

There will never be agreement what's the set of distinct characters (also, what characters should be included, bitcoin logo, facebook logo?)). I see Unicode as a necessary evil. Due to its complexity most applications should treat Unicode text as black boxes.

I never rely on Unicode for computation. When receiving Unicode I always make sure it's in the ASCII range. It could be argued that there should never have been Unicode domain names but I guess Western people are very lucky that ASCII includes most of their characters...

One goal of Unicode has been lossless round-tripping between legacy encodings (to encourage adoption). If such an encoding contains both Latin and Cyrillic, they must be separate to enable that.

The (seemingly obstinate) answer is that they are different characters. The Russian H sounds like an N in English.

If you're transcribing a conversation at the UN and there is a mix of different languages the fact that "Het" is transcribed as a latin character set is information. Het may be a southern American group of people, or it could just be a Russian dude saying "no", even if it looks the same.

I understand that we're still burdened by intralanguage homonyms, but I appreciate the fact that it isn't complicated further.

the font metrics and hinting/kerning are likely language or dialect-specific

Compatibility with ISO8859. For example, for Cyrillic, the first 128 characters U+40xx match ISO8859-5.

>ilovevitaly.com

This Vitaly guy…

I got tons of referral header spam (that shows up in e.g. Google Analytics) for all sorts of social media buttons and EU cookie law scare tactic sites. And then there was Vitaly who just spammed me with ilovevitaly.com, which if I recall correctly actually was a site about himself at the time.

Interesting, this domain now redirects to:

    http://money.get.away.get.a.good.job.with.more.pay.and.you.are.okay.money.it.is.a.gas.grab.that.cash.with.both.hands.and.make.a.stash.new.car.caviar.four.star.daydream.think.i.ll.buy.me.a.football.team.money.get.back.i.am.alright.jack.ilovevitaly.com/#.keep.off.my.stack.money.it.is.a.hit.do.not.give.me.that.do.goody.good.bullshit.i.am.in.the.hi.fidelity.first.class.travelling.set.and.i.think.i.need.a.lear.jet.money.it.is.a.secret.%C9%A2oogle.com/#.share.it.fairly.but.dont.take.a.slice.of.my.pie.money.so.they.say.is.the.root.of.all.evil.today.but.if.you.ask.for.a.rise.it%27s.no.surprise.that.they.are.giving.none.and.secret.%C9%A2oogle.com

But think about the poor underrepresented folks using foreign character sets?

You really need to support this 'sub café {} café()' => Undefined subroutine café in your friendly and social programming language, otherwise you will be accused of discrimination. Of course the two é are not normalized.

Which unicode-friendly language does really check for mixed script confusables? Java only is my guess. Even perl6 falls into this trap.

http://unicode.org/reports/tr39/#Mixed_Script_Confusables

Slack is not doing anything. It's Google chrome filling up your clipboard with the "extended" version of the url.

The fact that we need application-specific security measures against this just emphasizes the problem. There are a lot of applications.

Why is everyone thinking so small? What about https://www.goоgle.com?

Or how about the word "gullible" isn't in the dictionary?

http://www.dictionary.com/browse/gulliblе

Browsers have supported this for almost a decade.

It's literally not their job to filter referrals... they do the opposite, they collect referrals.

I believe they aren't preventative measures, but responsive. So if Google contacts ICANN, then they may do something about it

Yes and no. One of the problems is that Verisign's handling of IDNs wasn't exactly the best conceived, which left them with silly IDN codepoint tables like this: https://www.iana.org/domains/idn-tables/tables/com_latn_1.2....

That defeats the purpose of an internationalized dns system.

The whole point of getting unicode into domain names is so we can have 新浪首页.com so that it's no longer a latin alphabet centric system.

Meh. Markup ate my "radioactive pie" (9762 dec / 2622 hex) symbol :-(

Technically, Unicode is only one character set. If you want to disallow mixing, you have to disallow it on some other basis, like script. There are many edge cases to consider, though, and many legitimate reasons to mix scripts.