A few dialects are unintelligible indeed, and they're mostly spoken in southern parts of China. But most people can understand dialects spoken in central and east (north of Changjiang river) fairly easily even though they cannot speak those dialects.