Configuring OpenSSH's ControlMaster setting here makes an enormous difference. Summary: it keeps a connection open for a while in case you want to connect to the same machine again. If you do, it reuses that connection so the new one is nearly instant.

Tramp offers different "connection methods" with different characteristics. For example, the scp connection method uses scp to copy files to/from the remote machine, and this implies a new ssh connection each time.

But the ssh connection method transfers the files inline, using base64 or uu encoding, and then you do not need a new connection each time the file is read or written.