I just realized that the button is impossible to interact with on Monterey. The only way to get the "options" drop down (something that used to be individual buttons on the notification) to appear is to hover over the "X" but if you move the mouse away from that the options drop down disappears. There's actually no way to do something like snooze a notification from the notification itself.

It's an astonishing regression in functionality, even for Apple. And with Apple's recent UI "improvements" I'm not sure if it is a bug or deliberate. I'm probably holding it wrong but I can't figure out the combination of interactions to get that button to persist.