> you carry the responsibility for it's output (not in a legal sense obviously)
why not in a legal sense? If someone asks me what cleaning supplies are safe to mix, and i just ask some chatbot, don't vet the output, copy the response, and they end up poisoning themselves, am I not responsible?
If I'm a lawyer, and pass unvetted AI legal advice to my client, and they go to jail, should i keep my license?