It is a bit more nuanced than that, on both sides, you should always look at the full situation.
There's many things, like stability, and how the firm treats you that SHOULD be considered. But they should be considered, as benefits, and given a value.
I faced this on my last job search. I ended up declining a FAANG to goto a small firm. No regrets when the FAANG laid off a few months later like I predicted they would.
... Always watch out for yourself. Money isn't the only thing that matters. Relationships matter etc.
But as always: Business, is business, nothing personal.
