Debian is famous for modifying all programs it ships, it is more the rule than the exception. That's the deal I get when choosing Debian. SSH is more of a protocol, than a trademarked program.
> Organizarions that open source software can have closed source infrastructure that you can't request.
Which can't be a source for the program binaries, so you can still audit them, you just can't rely on e.g. their proprietary test suite.