One possibility for speed as in latency would be to pre-agree on a latency budget (as in realtime-systems: if you exceed that deadline, your system has failed). Then have everyone be aware of how they spend that latency budget. Say the latency deadline is at 500ms for your website to full interactivity. Currently you are at 320ms. Marketing wants to include some analytics scripts. Include them in the test page, measure the added latency, then check against your deadline: added 200ms, we are now at 520ms. Do we reject marketing's wishes or do we make the design dept. cut back on their image load times, maybe they can get from 130ms to 90ms? How about investing in better caching to get 100ms?

That way you can discuss numbers and can quantify how something impacts the overall experience. Budgets is something everyone can understand, and taking a big gulp out of a limited budget is something no-one wants to be seen doing.