In C# you can encode/decode strings to byte arrays based on your desired encoding, but a string is composed of characters, it's in memory representation is abstracted.
Is this a performance or zero copy thing? Not having to encode/decode to get to the bytes?
That's a hard problem, and avoiding it in every situation would require scanning the strings for surrogates beforehand, when you might never need to know that information. Go makes it explicit that knowing the exact character position and string length in characters comes at a cost.
There's a good discussion of this on Tim Bray's blog: http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF
Note that many languages that appear to offer indexing by rune (e.g., Java) do not in fact do so, since their 16-bit "character" type is incapable of representing all runes. The fact that this is only rarely an issue points at the fundamental rarity with with code needs to deal with runes-qua-runes.
Other languages like C# seem to be different on the surface, but in fact they index and measure by code units as well (2 byte UTF-16 code units), not by code points.
You can't usefully index a unicode stream in constant time and do correct and useful textual stuff anyway due to combining codepoints which may not have precombined forms (if only because there is no defined limit to the number of combining codepoints tacked onto the base) (so normalization will not save you) or codepoints which are not visible to the user and which you may or may not want to see depending on the work you're doing.
People really need to come to terms that a unicode stream is exactly that, a stream.
To find an index of a substring you need to scan the string, right. But once you have the byte index you can quickly jump to its position in the string, e.g. when you do a slice operation based on that index: s[i:]. If strings.Index() returned a code point index and not a byte index you would have to scan the string again.
How about indexing, measuring and slicing operations based on user-perceived characters, then?
There is a difference between a string and a byte array. A string is a string. A byte array is a []byte (byte slice). You have to explicitly cast from one to another. Neither are inherently utf8. A string is represented by a byte array under-the-hood, and string literals in your code are read as utf8 encoded. Strings themselves are not necessarily utf8 encoded, and if you need to use a different encoding there's libraries for that (unless you're using something really esoteric).
The distinction between a string with one encoding and a string with another is subtle but vitally important - exactly the sort of thing a type system should take care of.
The `next(string, index)` function used for the iteration protocol works like the `utf8.DecodeRuneInString()` shown in the example, but it returns the next valid index rather than the character width.
(check the docs, there's also RuneStart, which returns true/false if you index mid-rune)
