I take it you don’t have ADHD? The reason people with ADHD often work while wearing noise-cancelling headphones even with no audio playing is because this mechanism doesn’t properly work, it can’t tune out
anything.
As result, we need to minimize external stimuli to be able to concentrate. And being uncomfortable due to the cold is one such stimulus.