You got my hopes up but it seems I'm stuck paying UK VAT even though I don't think it's necessary any more.
I've run into this with a few different German businesses, I don't know if I'm right or they are, but it seems like they don't _have_ to charge me VAT. Which is underlined by the huge "import VAT" bill I'm dumped with when I have physical goods delivered to me in the UK.