If you choose to view the universe in one particular coordinate system (e.g. one centred on your eyes), then you'll see some things mysteriously happening. For example, you might find that if you release something from your hand, it will mysteriously move towards your feet. Eventually you'll realise that a bunch of things can cause things to mysteriously move, and you'll start using the word "force" to describe "tendency to mysteriously move".

But some of these forces are not observed by other people who are watching; they arise as by-products of the fact that you are the one doing the measuring. For example, if you're falling in a lift, you won't observe the force that causes a ball to fall towards your feet; but I, standing on the ground outside the lift, will observe that actually you and the ball and the lift are subject to this force.

In fact there's an underlying reality which we can't directly observe but which we can infer from the paths of objects. That reality is a curved spacetime, not a flat one (as it appears to be). This curvature means that "straight line" is actually not quite what you're used to; things follow straight lines, but those straight lines don't look straight to us, because of the underlying space's curvature: we can't see the whole of spacetime, only small segments of it, so we can't see enough to get a proper sense of the curvature. But since our limited frames of reference have their own notion of "straight line" which is incompatible with that of the global spacetime, we observe a mysterious tendency of things to deviate from what we think is the straight line they should be following.

So gravity "is a force": it's a mysterious tendency of things to move, because we are limited in what we can see and our own observation frames are subtly incompatible with the global structure. But it's also "not a force": if we were somehow able to take a fully global view of the universe, there would be no mysterious movement, only a huge number of things moving at exactly the same speed in perfect straight lines through a curved spacetime. (Assuming General Relativity is 100% accurate.)