If you had a magic oracle and, say, bought a ball for the fair price of $1 the day before you knew it would spike to $100, you are denying the seller an honest opportunity to sell it for $100. It's the moral equivalent of keeping your mouth shut when a clerk rings up an item for less than it costs. It might not be illegal, but it's certainly not "creating value" - you're using an information disparity to create a transaction with someone who wouldn't agree to your terms if they knew what you knew. ("Fools and their money" or something along those lines.)

Of course, once you add in the realities of uncertainty and much longer time periods, you can make more subtle arguments about risk exposure, hedging, etc. But in the simplest case, trading on privileged information is absolutely zero-sum, at least in the sense that there's a clear winner and loser (and in the stock market, it's illegal!)