Depends; etymonline says this:
> from Greek elephas (genitive elephantos) "elephant; ivory," probably from a non-Indo-European language, likely via Phoenician (compare Hamitic elu "elephant," source of the word for it in many Semitic languages, or possibly from Sanskrit ibhah "elephant").
So the Greek word is not necessarily native to Greek.