Yup: see https://hackage.haskell.org/package/ghc-internal-9.1001.0/do... where it is clear that setting a handler simply writes to an array inside an MVar. And when the signal handler is run, the runtime starts a green thread to run it, which means user Haskell code does not need to worry about signal handler safe functions at all, since from the OS perspective the signal handler has returned. The user handler function simply runs as a new green thread independent of other threads.
But I like the fact that you brought up this idea. Haskell can't do it but in a parallel universe if there were another language with no runtime but with monads, we can actually solve this.