It's just as bad to click a button you don't want to, when you're merely trying to scroll, or highlight text, or click another button.
Making a button's clickable area larger than the button itself can be desirable in limited circumstances, but it's not a general-purpose solution.
I also recall an obscure fix necessary for Mobile Safari. If you wanted something clickable near an <input> you needed to take care how you designed it because otherwise the <input> would steal the click. You could add an onclick handler (Mobile Safari has a bunch of heuristic hacks where onclick= will cause differences to touch behaviours when a touch is on an element that has a click handler. You could also use a button which might or might not be more accessible depending on needs and design. I recall I needed a lot of fiddling depending on each control.
Argue with Apple because Mobile Safari makes a tap close to a button click the button (and it causes exactly the problems you've predicted, and workarounds are difficult). Do you do a lot of close testing?? Because the feature is quite noticeable.
Try it yourself on an iPhone (ideally use something that can do smaller taps than a finger, with zoom and without zoom): https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...
I recall that similar features are more obvious on Android because you can make taps visible.
Virtual keyboards also have interesting responses to close taps on key buttons.
Ex: a.fn-link { padding:10px 18px; margin:-10px -18px; }
You can set a (pseudo) element as an invisible enlargement of that button but then you will get accidental taps.
It's actually a credit to google that you can scale the fonts up so much and then forget you had done it. In the old days, the UI would be broken in various places.