Phones consume a miniscule amount of power compared to other things you probably use. Even if it is 20% efficient if is still a drop in the ocean compared to air conditioners, tumble driers, fridges, ovens, kettles, toasters, hair driers, etc.
Small changes are still small when you "multiply them by millions" because all the big things get multiplied too.