In my experience its's less about caring what people specifically think of you, and more about how you are treated in whatever team you're on.
Example: I made a trade from a key role to a dev role in one job change. In dev role I was working with a technology I had significant experience with in my prior "key" role. In the new role, when the topic came up, I suggested we avoid certain vendors since in my prior role I was responsible for the technology vetting and I determined the chosen vendor was not forthcoming with some important limitations. My feedback (constructively and succintly presented) was completely ignored since I was "just a developer" and I ended up having to make the sub-par technology actually work.
It was frustrating and painful, not because of ego, but going into the task knowing it would suck since the wrong decision was made and now I was the guy having to make broken pieces meet some demanding requirements because "a developer" couldn't possibly know more than an "architect".