You have taken on the work the distributions do in the open source world. No upstream open source developer takes that on. Instead of getting bug reports from users, upstream developers insist bug reports are filtered by the distro maintainer first. They fix problems on their side so you never see them, and the ones that do make it through have been triaged. It's a win for everyone.
So the solution is to handle it the way they do. Choose a couple of baselines: maybe Debian Stable and Fedora. Publish packages for them, and make it plain they are only certified for those platforms. Make the rest someone else's problem: if you want it working on distro X, you package it for distro X. You've done the bulk of the work for them anyway, as most of them are either Debian or RPM based.