In the US a baseband processor's entire software stack that controls the radio front end must be certified. They'll also have the modems to talk to the cellular networks. BPs use their own CPU(s) and an RTOS firmware that's FCC certified.

This is why a baseband processor is a fully separate component from a device's application processor(s). Since the AP doesn't talk directly to the radio it doesn't need to be certified and can be updated without recertification. The BP can also get certification and any manufacturer using that BP doesn't need to re-certify it. The interfaces are also such that the AP can't (or shouldn't be able to) tell the BP firmware to boost the output power above legal limits or something.

Radios that have "open" soft modems don't typically have fully software controlled radio front ends. The radio front end will have its statutory limits baked in electrically or have very limited software control. The modulation on the back end isn't as important as the front end. Broken modulation just means you can't talk to anyone, an overdriven transmitter is effectively a radio jammer or can give someone an RF burn.