One of the actual reasons is that drivers go through intensive training (er...) as first-call engineers.

If a fault stops a train, it's far more useful to have someone on the scene to try emergency repairs than to wait until the train can be cleared from a tunnel - because a stopped train can literally hold up the commute of millions of people.

There are also safety issues. Drivers are trained in passenger management, and every so often they need that training to deal with fights, illness, suicides, or all the other messy things that happen on a public transport system - the public part of that being at least as important as the transport part.

The line is equipped with an Automatic Train Operation system (ATO); the train operator (driver) closes the train doors and presses a pair of "start" buttons, and if the way ahead is clear, the ATO drives the train at a safe speed to the next station and stops it there. This system has operated since the line opened in 1968, making the Victoria line the world's first full-scale automatic railway.[20]

http://en.wikipedia.org/wiki/Victoria_line

Platform edge doors is definitely part of it - people get stuck in the gap, children fall through, there's few practical driverless systems that can cope with that. Yet there are some places - e.g. Central line - where the gap is almost a foot between train and platform. The doors aren't the only problem, it's the whole carriage, on those curved sections.

Only one Union: The mighty RMT http://en.wikipedia.org/wiki/National_Union_of_Rail,_Maritim...

it was until recently headed by the late great Bob Crow http://www.bbc.co.uk/news/uk-26527325

RMT (the union) mainly (there may be actual technical reasons too for some lines, but any technical reasons are likely to largely be a result of knowing that there's no point in investing too aggressively in enabling full automation if they won't be able to make full use of it). Notice the page takes care to point out that current drivers jobs are safe.

It's likely that London Underground is basically betting it'll be easier to just hire at less than the replacement rate and gradually automate more and more rather than risk massive conflict with the RMT for years.