It is pumped, but the pumps are there to force circulation and keep things under control rather than to provide the pressure directly.

You need a steady gradient of pressure. The pressure you need at the bottom of the hole will fracture the rock in the shallow section. Pumps add a constant pressure, so there's no way to get the right pressures at each depth with a pump.

Instead, drilling exploits gravity to do the hard work. The drilling mud is very dense (almost as dense as the rock) so that the pressure at the bottom of the borehole from the weight of the mud is just a bit less than the pressure from the weight of the rocks.