They are stored encrypted but whether Apple has the key depends on whether you've turned on "Advanced Data Protection" (aka "I don't expect Apple to bail me out when I lose access to all my devices"). The table in this support article details the treatment of various data categories under the two options:

https://support.apple.com/en-us/102651

The default for many categories is that your keys are in iCloud so Apple can recover them for you. With Advanced turned on, the keys are only on your personal devices. A few categories, like the keychain, are always only on your devices.

Specifically, see Note 3: "If you use both iCloud Backup and Messages in iCloud, your backup includes a copy of the Messages in iCloud encryption key to help you recover your data." Under normal protection, Apple has the key to your backups, but with Advanced they don't.