I think the answer is that it depends. Software engineering is incredibly hard if you are a perfectionist who wants to make efficient, secure and maintainable software. But most probably it is not even possible to be that perfectionist and stay within any given budget. And the requirements are most often not your own.

The thing is, it's a job that needs creativity, spontaneous decision making as well as personal responsibility for those decisions. It's a real easy job if you don't need to take this responsibility (e.g. those who come after me when I am long gone have to deal with the consequences). It becomes a hard job the instant that you have some passion or ethical concerns that drive you to create software that holds up to your own high standards and requirements.

I think that's what makes it so hard for many. We are incredibly passionate (why would we be on this forum in our free time otherwise) but we constantly have to betray our own principles to make it work or stay employed.