Historically the backspace key is literally that, moving the typewriter's carriage back (when the space key moves it forward).

On current computers the space key now inserts a "space" character instead of simply moving the cursor a step right.

Backspace isn't symmetric to that anymore, as it doesn't even insert a space back but deletes backwards, so it is (pedantically) a misnomer.

And this is true for "modern" computers, older ones were closer to teletypes and would move back for overwrite (e.g try vi compat mode).