They are indeed based on similar code in the training dataset, but it also takes into account the context of the code in the conversation thread. While the initial code it generates is already impressive and pretty accurate. To me, the real power comes out when iterating on the code through followup conversation.

For example, I asked it to write an implementation for a piece of Kotlin code I provided describing an interface for a CRUD TaskRepository with typical `addTask`, `removeTask`, `getTask(id: UUID): Task?`, `getTasks(parentId: UUID): List<Task>` methods. It generated a TaskRepositoryImpl class backed by a `Map<UUID, Task>` and all method implementations where correct, including checking a projectId property in the getTasks method. Note that I never gave it the code for the Task class.

Then I iterated on it by asking "Add save and load methods that store state in a JSON file" and it did just that: Introducing the Gson library, serializing the backing map to a JSON string before writing to the file, loading the file again and deserializing into the map.

After that I asked "Add a SharedFlow<TaskEvent> that publishes events when creating, removing and updating tasks" and it was able to infer where to emit events, providing sensible names like `TaskEvent.Created`.

In another test case I asked it to write an Android layout xml for a login screen with username, password, login button and login spinner which it did. I iterated once asking it to use components from the Material library and to make sure the Spinner was overlaid over the form. Then I followed up with "Write the corresponding activity code in Kotlin". It generated a full LoginActivity class that loads the layout xml, findViewById for the fields and buttons, input validation with error handling, show/hide the login spinner when logging in, starting the MainActivity when login succeeds and it even mocked the login method with an artificial delay of 2 seconds. Its explanation of the code described that the delay was there to simulate a login, while in real code you would replace this with an HTTP call or a call to a loginService.

And have a look at these snippets of conversation where I asked it to improve a dueLabelString method I provided. https://imgur.com/a/ZtViC3d

Is this "reasoning about the code"? I don't know and it's hard to say.