I once debugged a pathological case of this where OnApp would configure Xen with a CPU limit of 200% on a 24 core machine for Dom0, so it could only spend 1/12th of the time in Dom0 - this is measured in time spent but also applies to all 24 cores. It would more or less totally lock the machine because 1 core was waiting on a spinlock or similar from another core, and they just scheduled so slowly it took FOREVER. Combine that with minimimum timeslices of 30ms.. basically each 30ms if would spinlock and do nothing useful. Good times.
Even their support team couldn't figure that out despite having other customers running into it, I managed to figure it out using xenalyze or something basically tracing the scheduler actions and found that all the dom0 cores were only being scheduled something like once per second each. Was kinda crazy.
No Batch Core scheduling in that version of Xen either. I think newer versions might have it? Might not work when you have every core assigned to a domain though.
The solution was to only plug 2 CPUs into Dom0, rather than plug all 24 and give it a 200% slice of them.
