> It is the observer at infinity that never sees you fall into the black hole
We don't even need an observer to be at infinity, thanks to the expansion of the universe. With some future telescope our descendants may observe something on a trajectory to enter a black hole in an early-universe galaxy that is just crossing that observer's (cosmological) horizon.
I think it's relevant to raise this since the article at the top is about embedding black-hole-like collapsed stars in an expanding universe and the research which directly discusses the observable consequences.
> real physics is local
Yes, absolutely. You still get spaghettified if you fly into a black hole which is the only other appreciable mass left in the far far future of our universe. Nobody needs to see your last moments.
> you have to use the coordinate system of the person falling into the black hole to determine what happens to them
No, you can use any coordinates you want (or no coordinates at all), but you have to be aware that there are quantities which are invariant under changes of coordinates (e.g. the curvature scalars) and quantities which are coordinate-dependent, and that some systems of coordinates make the latter difficult or even impossible to calculate.
Indeed the infaller can use any set of coordinates she or he wants. Some time coordinate (wristwatch? distant pulsars?) and spatial spherical coordinates with the infaller always at the spatial orgin, East-North-Up coordinates originating on the (spinning) black hole, etc. are all (pardon the pun) attractive in these circumstances.
Also, defining exactly where "falling in" happens is tricky, even for the infaller. Visser 2014 on horizons: <https://arxiv.org/abs/1407.7295>, second sentence third paragraph of the Introduction section ("These distinctions even make a difference when precisely defining what a "black hole" is -- the usual definition in terms of an event horizon is mathematically clean, leading to many lovely theorems [20], but bears little to no resemblance to anything a physicist could actually measure.")