They may load async but JS parse + compile + execute takes up client device CPU cycles and can block the main thread from responding back to user actions immediately. More so on mobile devices. This, and the fact that a typical page has multiple such trackers can cause notable overhead.
Can you share some example sites where the CPU cycles on mobile are a major impact? I suspect maybe news websites? I just don't encounter many on a daily basis where it has an impact and most sites I look into for performance - the issue definitely isn't trackers. It's almost always oversized large css files and blocking js files. When you dig into the site you usually find that it will be very challenging to architect it to avoid the blocking script or shrink the css... but again would love to see some "typical" examples.