Right, but TLS1.3 improves that especially with 0RTT. Before that you had things like session resumption for repeat clients, or if your server was overloaded you could use an external HTTPS proxy.