The main ways to protect against regressions that I know about are 1) having a large set of (mostly integration-level, for me) automated tests, 2) adding business metric monitors, and 3) periodic manual human QA. Wondering if there's anything I'm missing.
With a growing codebase, the number of tests written gets almost out of control, and it's unclear which are the important ones. I've also seen business metrics being recorded in a place that "misses a spot" or otherwise breaks independently of the actual thing it's measuring, too. And there's no way to know the human flows are operating at the right granularity.
Is this something everyone just figures out ad hoc? What am I missing?
Does your company have strict requirements around this type of thing?
Does the decision change if it's for iOS vs. Android?
In short, if you were to build a new SDK today, how hard would you fight to make the SDK open source from day 1?