Phone screen sizes grew as the applications that could use screen space grew in demand.
People are watching 1080p films on the train now. The people who want smaller screens are usually willing to deal with a larger one. People who want larger screens usually cant operate their use cases on a smaller screen. Larger screens also tend to mask larger case meaning less miniaturisation required for the components.