At the same time, the tools which solve this really shine. You inevitably run into these issues with random third party dependencies on other platforms, too, but it's further from the norm, so you end up with awful bespoke toolchains that contain instructions like "download Egub35.3.zip and extract it to C:\Users\brian\libs."
Developers on GNOME deal with this regularly, partially because of our lack of a high level SDK. So one of the tools we have to solve this is first class support for flatpak runtimes in GNOME Builder: plop a flatpak manifest in your project directory (including whatever base runtime you choose and all your weird extra libraries) and Builder will use that as an environment to build and run the thing for development. This is why pretty much everything under https://gitlab.gnome.org/GNOME has a flatpak runtime attached. It's a young IDE, but that feature alone makes it incredible to work with.