We could continue raping that metaphor, or we could just discuss whether or not putting ssh on a different port actually accomplishes anything useful.
For example: it's generally a pretty safe assumption that most remote servers have ssh running on them somewhere. If you are running scripts to find vulnerable or misconfigured ssh hosts, then it makes sense to scan the standard port and try stupid logins. In that case, if the sysadmin moves ssh to a different port, then they "protect" themselves from the various scripts out there, but then again simply having reasonable password policy accomplishes the same thing and without the hassle of having to locate ssh on a different port.
On the other hand, if you're a dedicated attacker and you want to compromise a particular system, then moving ssh to another port does squat; nmap will find it easily enough. Port knocking and/or denyhosts and/or iptables or pf will stop that easily enough, but then again, a dedicated attacker probably won't try ssh first anyway, since that's hard. Attacking something else like your web app or any of your other services, would make a lot more sense.
So, either way, running ssh on a different port is completely superfluous. While you haven't "made the armor any weaker", you also haven't made it any stronger. At all.
Because ssh is not a tank.
It costs something to use a port knocker. It costs something to use a nonstandard port. If you have strong authentication (say, using 2048-bit ssh certificates), it's probably not worth paying those costs. Maybe it is if you're a bank, but probably not if it's the server that hosts your blog. If you don't have that strength of authentication, it would be better to pay for the better authentication than for the port knocker and nonstandard port.