Maybe that's what they claim to do, but how do you know for sure? How do you test for it?
> This implies that this is the only or main way security is achieved.
It doesn't, there are many facets of security, social engineering being one of them. Maybe it's controversial to test something that requires misleading people, but realistically the only alternative is to ignore the problem. I prefer not to do that.
> Plenty of organizations agree to probing/intrusion attempts; there is no reason to secretly do this.
Yes there is: Suppose you use some company's service and they refuse to cooperate in regards to pentesting: The "goody two-shoes" type of person just gives up. The "hacker type" puts on their grey hat and plays some golf. Is that unethical? What if they expose some massive flaw that affects millions of unwitting people?
> I don't believe in the ends justify the means argument.
Not all ends justify all means, but some ends do justify some means. In fact, if it's a justification to some means, it's almost certainly an end.
> I do agree that the way the current laws regarding "hacking" are badly worded and very punitive, but crimes are crimes.
Tautologically speaking, crimes are indeed crimes, but what are you trying to say here? Just because it's a crime doesn't mean it is unethical. Sometimes, not performing a crime is unethical.
> You don't randomly walk up to your local business with a lock picking kit to "test their security".
Yes, but only because that's illegal, not because it is unethical.
> You don't randomly steal someone's wallet to "test their security".
Again, there's nothing morally wrong with "stealing" someone's wallet and then giving it back to them. Better I do it than some pickpocket. I have been tempted on numerous occasions to do exactly that, but it's rather hard explaining yourself in such a situation...
> Why is the digital space any different?
Because the risk of running into a physical altercation is quite low, as is the risk of getting arrested.