There is a disclaimer, yes, but you have to admit that it's pretty shit, innit? I mean for one, it's about the size of a human hair, and at least when I tried it, the disclaimer came up only when I clicked the "Show More" button. It might admittedly show up earlier if the response is shorter, admittedly I don't know. Also personally I'm a bit uneasy with the idea that just with a simple disclaimer they could avoid any and all liability. Not your argument, I know, but still.
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.