I've learned that there are people who don't have this response, they simply keep it on an emotional level. These people also never experienced or saw any physical fight during childhood and were never really bullied.
I saw fights pretty much daily as a kid and experienced them at least once per month, winning about 50% of them. So I've experienced both sides quite a bit.
I've noticed that kids who fought daily have this much more in their physiology than I do.
I was thinking more about people who grow up with very little, i.e. aren't sure if their parents are going to put food on the table that day. If you start out like that but manage to get a steady job etc..., I think it can be much harder to take risks like changing career or being assertive in certain situations.
It’s more of an internal preservation thing than a fear of externalities.