If you argue based on English, you should point only to the ordinals "third", "fourth" and the like.
In the older European languages, the initial position in a sequence was named using words like "first", which mean "closest to the front". The next position was named with words meaning "the other", e.g. "alter" in Latin (the old European languages had 2 distinct words for "the other from two" and "another one from many", e.g. "alter/alius" in Latin).
However when the need for naming other positions in a sequence besides the first, the second, the last or the next to the last (penultimate) has appeared, then the ordinals derived from numbers have been invented, like third, fourth etc.
In later Latin, the word meaning "the other" has been replaced with a word meaning "the following". This has been inherited in the Romance languages and it has been borrowed in English as "second". Also in Old English, "other" had been used for "second", before "second" was taken from French.