As an anecdote, I actually had a fair bit of trouble with geometry in high school even though I did very well throughout high school in math/science generally and went on to major in engineering in college.
I'm not so sure about the visual thinking part but wrt symbolic representations at least you're probably right as I've never felt a particular connection to higher level math and theoretical physics.