Decoupling capacitors don't remove the ground reference, they just allow high-frequency signals a faster path to ground.

Typically, you need dedicated circuitry (and usually inductive coupling) to provide full isolation, but if the circuit is using this layout then you can still choose to ground the normally-isolated side for probing.

"the ground" ≠ "ground". It's unfortunate naming IMO, but "ground" is just the 0V reference point. A normal oscilloscope's ground probes are mains Earth referenced, that is they're connected to the "ground" pin in the outlet which in turn connects to one or more conductive rods buried in the Earth. Decoupling capacitors don't negate the ground reference, since they're connected between a power rail & circuit ground. So the power rail's voltage is still referenced to ground.

A truly "floating" circuit would be something battery powered, or galvanically isolated from mains Earth (e.g. by an isolation transformer).