The interesting part is what to do when ChatGPT does it wrong, and if the problem is not trivial and an exact solution is not available online, it is usually wrong. Sometimes, not by much, but it takes skill to notice the problem and fix it, either manually or by asking ChatGPT for it.

Same idea as for libraries. One could argue that those using the "sort" function don't know how to write a sorting algorithm, but in real life, unless there are particularly good reasons, rewriting a "sort" function would be crazy, and probably not what you want from your employees.

If you want your candidate not to use the tools at his disposal, you may frame it as a requirement. "on this test, imagine you are working on a sensitive project, you are not allowed to upload any details to a third party service, you can search the internet for generic information, but do as if your development machine has no internet access, so no copy-pasting". Or, for libraries "on this test, imagine we absolutely want to limit dependencies on third party libraries, even if it means sometimes reinventing the wheel, so only <list of libraries> are allowed".