When fixing performance problems you shouldn't guess, just profile it to find the bottlenecks.
I've seen plenty of performance 'fixes' that weren't, pure guesses by developers that did nothing, when a quick profile immediately revealed the culprit.
In your case you also need to figure out if it's happening server-side or client-side. I generally start with the server-side logs, get a few days/weeks worth of data, find average page request times, plus how much deviation on those requests, then go from there. That gets you the server-side. For client-side, unfortunately it's a lot harder. Google analytics page load speed, for example, is a pile of crap. But, again, there's a profiler in dev tools, remember js compile time is a significant thing and can slow load time too so check that out as well as the actual run times (js compile time shows in the page load graph).