1
What I want is for it to use asserts instead of optionals, for it to let errors propagate.
What I have tried so far: - Specify my preferences in AGENTS.md file many times, yet it doesnt seem to make any difference in this aspect. - The rest of my code base does use asserts, and does not have unnecessary optionals, and yet it doesn't pick up that pattern.
What does work is that once it's done coding, I specifically tell it to go through the code and remove unnecessary optionals, and add asserts instead of defensive if conditions. But here too, I have to tell it to do so multiple times.
Any ideas to solve this would help. Thanks!
I am looking for a service or effective technique using which I can get quick and reliable translation of our Node SDK (https://github.com/supertokens/supertokens-node) applied to our Python (https://github.com/supertokens/supertokens-python) and Golang (https://github.com/supertokens/supertokens-golang) SDK.
Our SDKs are not just wrappers around an OpenAPI spec, so we can’t use existing tools to auto generate our backend SDK. Or even if we did, that would only generate a very tiny percentage of our SDK. Other than being API wrappers, our SDKs:
- Provide several overridable functions for users to hook into.
- Manage reading from the request and writing to the response objects of various web frameworks of these languages.
- Expose APIs to via a middleware (that again integrates into various web frameworks). Each API has a lot of logic that includes input checking, business logic of that API, calling various other APIs to do actions like send emails, and finally writing an output JSON.
Currently, we are hand writing each of these SDKs, and as you may imagine, it is very expensive. As a result, we have the Node SDK (which is our most used one) far ahead in terms of features compared to our other two SDKs.
As an example of the set of changes that need to be replicated in the other SDKs, have a look at this PR: https://github.com/supertokens/supertokens-node/pull/670/files (many of the files are build files which can be ignored, but even then, it’s 200+ files changed, a large chunk of which are adding tests).
What I have already tried
I have primarily played around with GPT-4 with different types of prompts for simpler PR changes (for example this PR: https://github.com/supertokens/supertokens-node/pull/782/files).
For the TS code changes, I tried a prompt that gave the raw git diff of the node PR and the contents of the python function in which the changes are to be applied, and then asked to generate python snippet with the changes, and the output was quite accurate.
For test case changes, I gave the existing python test file and added it to ask new test cases based on the diff, and that went quite well too (though not at all perfect - i had a bunch of false imports, or made up function names, but 90% of it was good).
This, of course, is a very simple PR, and it would have taken me lesser time to do it by hand than try with the prompts, but I wonder if there is a way to scale this to make it work (even if it’s 50-60% correct).
My ask
If anyone has experience with working with LLMs for this or a similar purpose, or if there is a service out there which can help me with this, I would love to be connected. You can email me on rishabh@supertokens.com.
Thank you!