Many languages don't correspond to flag ideograms, and many flags don't correspond to a single language.
Ideograms are fine so long as they can be replaced with a canonical word or phrase and still make sense.
Flags for languages fails that in many cases around the world- if I want to select Quebecoise French, neither the Canadian, French, or Quebec Province flags are appropriate.
Likewise, Switzerland has no official language, and many countries such as Belgium have multiple.