Considering it uses the same compiler infrastructure, it's mostly because of using higher level more expensive constructs. It can also generate code as fast as C++ in some cases (see the language benchmarks game).

>which incidentally also has everything you mention (mostly in the form of libraries to prevent bloating the language).

Well, hard to bloat C++ further anyway. It does have a better ecosystem and more mature compilers, but on a purely language level, I'd take Swift over C++ (including "modern C++" that still comes with all the historical baggage and a trillion gotchas) any day.