One usage of it that I find particularly annoying is in answer to questions. For instance:
"Where did you go to lunch?"
"So, we headed downtown..."
I think it's aggravating because it feels evasive, as if the person is pretending they're not answering a question and are instead just starting a new story.