Indeed, the scary thing is that there is no theoretical limit to how sophisticated a side channel attack could be. Imagine all the timing data that could in theory be gathered from html layout engines and css, even without javascript, just by resource loading times.
I would like to salute my shitty ISP for keeping me safe from timing attacks using their unreliable network infrastructure.