The polyfills haven't been slow since Polymer 0.8/1.0 when we switched away from the "full" Shadow DOM polyfill and to the much lighter-weight ShadyDOM shim (which trades a little spec compliance that Polymer hides, for speed).
Polymer demos, like for HNPWA, are consistently among the fastest, even beating the speed-focused React clones.