I'd invite you to watch the show Alone, where many survival experts have had to give up in the first day or two because they opted not to bring a fire steel and don't want to freeze.
You're right that it's possible, but how long it takes, and whether or not you can do it, are dependent on skill and environment. Wrong kind of wood? Screwed. Been raining for a day? Screwed. Sometimes this can be overcome, but you have to know what you're doing, and it can still take hours to generate enough heat to dry the wood until it's possible to friction combust it. Try it sometime, just bring a lighter with you.