Yes, most software is harmless. There are some scenarios where it could do harm, but ultimately said harm was the product of human usage of the software, not software inherently. There are a few examples such as medical device software and things like train software where honest mistakes can result in human harm. However even with those the issue was that humans made the mistakes.

Software development as a discipline certainly needs more rigor like the other engineering disciplines. However the AI act is premature since AI ultimately is an implementation detail for a variety of use-cases, that should be regulated independently, not AI. Regulating AI in general is like regulating electrons.

If the EU has an issue with AI potentially resulting in more misinformation, then regulate the types of sites, that is social media, that would be the vector for such spread.

I'm open to counterexamples outside those class of examples.