Moscow comes from the locative на Москви (na Moskvi), while the original form is *Москы. It's using the locative because the hydronym (name of the river) is older than the name of the city. So the city comes from the name of the river.

Further, Lith. mazgóti "wash, splash", Latin mergō "dive"

Yauza probably from ya- and -(v)uz (vyazat' - to tie, bind, knit, uzel - knot) so maybe something like "the river that ties"

There are several etymologies for Oka, it probably has a form of oko (eye) in it, maybe from a Baltic source.

Volga has the same root as vlaga, vologa - "wetness"