It adds about 2 minutes to our gitlab pipelines but the major issue with it is when organizations decide failures should prevent merging code to master or even deploying to a QA environment.

That's the real time sink - figuring out how to get past it. It's a lot more than 2 minutes, sometimes even days if it's something you can't work around and have to go through the red tape if your team isn't empowered to take charge of your own pipelines.