The same happens with foreigners learning Mandarin Chinese. The consonants are easy, but each vowel can be pronounced with one of 4 different tones (or 5 if you count the 'neutral' tone).
If you get a tone wrong, sometimes people will understand the erroneous word due to surrounding context. But pretty often you'll just elicit a blank stare. This is especially the case for short phrases, e.g. when you're asking for directions.
I wish.
My teachers have no problem telling apart my j and q, but I rarely hear the difference despite all the cosonants in my native language coming in palatalized/non-palatalized pairs.
