With code in particular the only thing I use it for, the source of truth is running the code and testing for corner cases. I usually know what the right answer is. It can just get to it faster.
I may not know always know the correct API or CloudFormation/CDK/Terraform syntax. But if it gets it wrong, I can read the docs and correct it.
Providing sources doesn’t usually help. ChatGPT consistently makes up sources.
Vanilla ChatGPT can hallucinate sources, but with the web browsing plugin in Plus it'll produce real links. Bard and Bing AI browse and produce accurate links right out of the box.
Certainly with code the proof is in the pudding, but most recently my problem was "I need to create AWS monitors in Datadog to alert when a region is down." ChatGPT was hopeless but bard was able to point me to the exact doc explaining how to set it up.
