Get it to explain something, lets say dynamic hedging for derivatives, and then ask it to explain how to exactly hedge something specific. Or describe some physical situation with a quirk and then let to derive the implications. Someone on HN had an example of asking to imagine entropy working in reverse in a cup of coffee with sugar dissolved. While it discussed sugar spontaneously forming crystals and other things, it never considered what the water would do, for example, let alone bigger issues such as if even the existence of water works etc.

Again, humans are often poor at these things, too, but if it had "mechanized" reasoning capabilities instead of "replicative" ones (i.e., just repeating stuff), I would expect it to do generally better.

Not OP but may have some idea of what they mean. Recently I had the following bizarre conversation with Bard. It appears to "know" what a word ladder is, and be able to explain it to me. On subsequent questioning it gives me a correct solution to a solvable problem (which I admit is impressive), a false solution to a non-solvable problem (whilst telling me that the solution obeys rules that it clearly doesn't), and then a bizarre "proof" that it's impossible to find a solution to the first problem I gave it (which it had already solved).

The 6-step supposed "solution" is the one that really gets me. It seems to have no idea that the constraints that it repeats in the same message have clearly been violated.

~~ME~~ What is a word ladder puzzle?

~~BARD~~ A word ladder puzzle is a word game in which you are given two words and must find a chain of other words to link the two, in which two adjacent words differ by one letter. For example, the following word ladder connects the words "cat" and "dog": cat -> cot -> cog -> dog

~~ME~~ Give me a word ladder which transforms the word FAWN to the word DEER in 7 steps.

~~BARD~~ Here is a 7-step word ladder that transforms the word "FAWN" to the word "DEER": FAWN PAWN PAWS PEWS SEWS SEES SEER DEER This word ladder is valid because each word differs from the previous word by exactly one letter. All of the words in the ladder are also valid words in the English language.

~~ME~~ Give me a word ladder which transforms the word FAWN to the word DEER in 6 steps.

~~BARD~~ Here is a 6-step word ladder that transforms the word "FAWN" to the word "DEER": FAWN PAWN PLAN PLAM DAME DEER This word ladder is valid because each word differs from the previous word by exactly one letter. All of the words in the ladder are also valid words in the English language.

~~ME~~ Is there a word ladder which transforms the word FAWN to the word DEER in 7 steps?

No, it is not possible to transform the word FAWN to the word DEER in 7 steps. The minimum number of steps required is 6.

Here is a proof by contradiction:

* Assume that there is a 7-step word ladder from FAWN to DEER.

* Each step in the word ladder must change exactly one letter.

* There are only 4 letters in the word FAWN.

* Therefore, at least one letter must be changed twice in the word ladder.

* However, this is not possible, because each step in the word ladder must change exactly one letter.

* Therefore, there cannot be a 7-step word ladder from FAWN to DEER.

I hope this explanation is helpful. Please let me know if you have any other questions.