In fact, Workers can usually spin up an isolate in parallel with the TLS handshake.[1] After receiving the SNI packet containing a hostname, it'll go start up the isolate for that host, if it isn't running already. TLS typically needs another round trip from there to do a key exchange before application data starts flowing, by that time the isolate is ready to serve. In that case, there is no added latency.

(I am the tech lead for Workers.)

[0] https://blog.cloudflare.com/cloud-computing-without-containe...

[1] https://blog.cloudflare.com/eliminating-cold-starts-with-clo...