Yeah, I just got ChatGPT to give me a recipe for chlorine gas that looks pretty accurate (per Googling, not a chemist). It took probably 10 seconds of prompt tweaking. A subsequent prompt gave me steps for purchase or synthesis of each material. I asked it for mustard gas, but the procedure looks very incorrect from Googling.