Explicitly, WLOG the potential is `V(x) = V_0 + V_1 (x-x_0) + V_2 (x-x_0)^2 + ...`. But the first derivative is 0 near a minimum, so V_1 = 0. The constant V_0 term doesn't affect dynamics, so choose it to be 0. Then the higher order terms go to 0 as x->x_0, so you have `V(x) = V_2 x^2` near any stable point.
And it's pretty obvious that lots of real-world phenomena 1. do have stable states and 2. do oscillate around them before settling, so the model is pretty good for lots of real-world systems (with the error being that the system is generally not conservative, so eventually it settles into the equilibrium because friction is stealing the energy).