I realized this (well, "realized"... I knew it, but it became very creepy) the other day, when my phone popped up a notification while I was in a restaurant, saying "enjoying <restaurant name>? Why not take some photos for people to see?" or something like that.
I've lately been getting into the habit of turning location services off unless I actually need them on, and I would recommend it. It's a shame my phone gets my location behind my back without any indication by default.