Even if it isn't 90%, most developers like writing code more than debugging, so most would prefer to automate the latter.
AI translating natural language into code probably isn't as important as AI generating bug-free code and/or debugging its code. Even GPT-4 struggles with this: sometimes you point out a bug and it works, but sometimes it just can't find the issue and starts hallucinating even more as it gets confused.
Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging...
Also, it's a bit ironic how one way to prevent bugs is using stronger type systems and formal methods. But, AI is particularly bad at formal methods. But maybe with a system like MCTS combined with much faster generation...
it was then maintained by at least 20 different people that had no idea what they were doing
very little of the altered logic makes any sense and 95% of the time consists of trying to reverse engineer what they were trying to do
then fix it up without breaking other parts of the codebase as many of the logic bugs interact with each other and end up cancelling out
with the garbage generated by LLMs... I suspect all development is going to turn into this
Sounds awful. I was hoping for the future where we would all be out of a job.
we had two new starters and one long term employee quit after being asked to work on that codebase
(and ended up having to do it myself at extremely short notice... worked though)
on edit: https://www.zippia.com/cobol-programmer-jobs/salary/#
the logic is simply broken
AI lay-person, web developer here.
This hits at something that's been nagging at me for a few weeks. LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news).
Deeper than this, "intelligence" isn't rooted in words. It's rooted in experience, logic and their interaction. The words are a side-effect; they're a way to label the experiences.
We're not training LLMs with experiences, only a reference to experience had by others, which may or may not map to the experience of the interlocutor. The LLM has no experience and limited/no logic to wrestle those experiences into wisdom.
Debugging is a "pursuit of wisdom" activity. Statistics, which underpin LLMs often identify truths (we wouldn't use them in science if they didn't), but they also hide truths. You need to clean data, take unusual slices of it, recombine it, and otherwise process it in order to uncover the true truths. I'm not sure LLMs can do that yet (if at all), and strongly doubt they can without _experience_ (sensory input, intention, failure etc).
It is not obvious to me that this is true.
The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token.
You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.
I think they only develop abstract understanding about the relationships of the words/group of words.
I think the word you are looking is knowledgeable. They have huge knowledge repository and can remix it in whatever way they please, but I don't think if you gave them a novel problem (outside their realm of knowledge) they could solve it.
As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its training set such as text -> sound.
If a technical product manager was able to generate the code needed for a feature using some combination of LLMs and CoPilots—even for proof of concepts, for example—the business would need to hire fewer developers and saves significant operating costs.
Extending it to your everyday person. Aunt Jane bought a new washing machine and wants to add it into her smart house. Either she's gotta call the niece to hook it all up, write the integration to Home Assistant, write automations for her iOS devices, etc, or, she asks her favorite LLM / Co-Pilot for help and it step-by-step walks her through it directly, with code, with individual troubleshooting.
There's business in both use cases, but the total addressable market is quite a bit bigger for one of them.
Your average person's ideal amount of exposure to code is zero (as in, most would prefer to not even think about code). If the device isn't automated from the getgo in an easy way, then Aunt Jane has no smart device and no matter how easy it is to write the connection, she's not going to have it unless her niece shows up.
You're severely overestimating the desire for an everyday person to automate their home. Most people are perfectly fine manually flicking light switches and unless you live on a very specific schedule, often automation ends up falling short of practical reality, so even the majority of people who "automate" just end up moving the light switch to their phone.
I know a cartographer who makes maps in some specialized software and then also has to occasionally interact with some official federally managed database. This person is not a software engineer and the database interaction has been historically painful in the office. This year ChatGPT just helped whip up some Python scripts.
Two hypotheses: 1) the “in house tooling” market is huge and will make great use of this tech; 2) that will contribute to pushing the floor lower and lower where more and more roles and people will take advantage of it.
This is exactly where it completely breaks down. The AI might as well show her some hieroglyphics.
Yes, today, developers are people who know programming languages, that's something non-developers don't know, so teach an AI do code and you can spare a developer, right?
But no, what developers really do is tell the computer precisely what it should do, a programming language simply the best way we have found to do it. Using GUIs, natural languages, etc... may fell less obscure to the layman, but the problem doesn't go away, in fact, it is often made worse because of the fuzziness of natural languages. There is a reason developer have been using programming languages for so long despite countless attempts to find alternatives, they simply haven't been matched for the task of telling a computer what to do precisely. And if something better is found, then developers will use that, and besides some syntactic technicalities, the job will be essentially the same.
bool
files_read(const char *path, char **data, size_t *size) {
FILE *f = fopen(path, "rb");
if (!f) {
return false;
}
fseek(f, 0, SEEK_END);
size_t n = (size_t)ftell(f);
rewind(f);
if (size) {
*size = n;
}
if (!data) {
goto ok;
}
*data = (char*)malloc(sizeof(char)*(n + 1));
if (fread(*data, 1, n, f) != n) {
free(*data);
return false;
}
*data[n] = '\0';
ok:
fclose(f);
return true;
}
> The issue in this code is related to memory access. The line data[n] = '\0'; should be (data)[n] = '\0';.
And then went on to generate corrected code and this explanation:
> The original expression, data[n], first applies the array subscript operation, which is equivalent to (data + n), and then tries to dereference the resulting pointer, which is not the intention here. This will likely lead to a segmentation fault because data is a pointer to pointer, and data[n] will try to access memory that has not been allocated.
> The corrected expression, (*data)[n], first dereferences data to get the allocated char array, and then applies the array subscript operation to set the null terminator at the correct location.
I'm not trying to say this isn't impressive, but seems fairly obvious and more like something a linter / analyzer would pickup ?
Actually, no, let's talk about the bug. What's with people's obsession of eliminating local temporaries and working straight with pointer-to-result?
char *buff = malloc(n + 1);
if (fread(buff, 1, n, f) != n) {
free(buff);
fclose(f);
return false;
}
buff[n] = '\0';
*data = buff;
ok:
fclose(f);
return true;
It kinda works tho. In my anecdote Copilot works much better with C# than Python simply because I can write the signature of a function and that it generates the content.
(I know Python has type annotations too, but Copilot just isn't as smart as with C#. Perhaps because there isn't enough training data in typed Python?)
But yes, what you described matches my experience. I'd say Copilot benefits those "tedious but relatively strict" languages the most.
But it obviously doesn't have knowledge of your domain, and I don't think any LLM could do a perfect job without that.
It happened so fast that so many people are in denial and many people aren't even asking the obvious questions. Simply ASK chatGPT to refactor your code to be more unit testable and IT Can do it.
Of course it clearly has your typical LLM problems but it is roughly 50% there. The game has changed. If AI gets to 100%, not only will the game be changed, but humans will no longer be part of the game.
See what I asked chatGPT:
ME:
def (x: int):
for i in range(x):
print(i)
Can you refactor the code so it can be more unit testable?
CHATGPT:
Certainly! To make the code more unit testable, we can refactor it into a function that returns a list of integers instead of directly printing them. Here's the refactored code:
python
def generate_number_list(x: int) -> List[int]:
number_list = []
for i in range(x):
number_list.append(i)
return number_list
In this refactored code, we've created a function called generate_number_list that takes an integer x as input. Instead of printing each number, we use a number_list variable to store the generated numbers by appending them in the loop. Finally, we return the number_list.
By returning the list instead of printing the numbers directly, we make the code more testable because we can now write unit tests that assert the correctness of the generated list.
IME GPT-4 can't write a bug-free 10 line shell script. It's particularly poor at inferring unstated requirements - or the need to elicit the same.
There's a general problem with LLMs: they're too eager to please. It shows up as confirmation bias. Embed a perspective in your prompt, and LLMs continue in the same vein.
You can, with careful prompting, try to provoke and prod the text generation into a more correct shape, but often it feels to me more like a game than productivity. I have to know the answer already to know how to ask the right questions and make the right corrections. So it feels like I'm supervising a child, and that I should be amazed it can do anything at all. And it is amazing; but for productivity outside tightly constrained environments (e.g. converting freeform dialogue into filling out a bureaucratic form - I think this is a close to ideal use case), I struggle to see it scaling up much, from what I've seen so far.
For creativity - e.g. making up a story for a child - it's not bad. One of my favourite use cases, after discovering how bad it is at writing code.
The game is designing software to requirements. It's writing literature for a new era. It's creating X for A audience with N vauge unspecified needs -- where X is a complex product made of many parts, involving many people, with shifting and changing problems/solutions/requirements.
The game was never writing the stack overflow answer -- that was already written.
instead it generates a load of redundant boilerplate
if I saw a developer check that in I'd think they were incompetent
It is pure pablum. It is almost the perfect example of LLMs producing fluent vapid bilge.
What I would like to do is feed in my entire 50k line program and get something out.
> ME:
> f(x) = exp(cos(x))
>
> what is f(0) ?
>
> GPT-3.5:
> f(0) = 1
Taking Nair's viewpoint, coding AI will be useful only to the extent that it assists programmers to build and employ their theory of the program. Or to itself develop a useful theory of a specific program, not just programs in general.
[1] https://gist.github.com/onlurking/fc5c81d18cfce9ff81bc968a7f...
I asked it to write a YAML parser in pure Rust. The first thing it did was write this:
pub enum YamlNode {
Scalar(String),
Sequence(Vec<YamlNode>),
Map(HashMap<String, YamlNode>),
}
Literally unusable as a starting point.
I did try to correct it but it tripped up and somehow managed to fuck that up to.
YAML allows YamlNodes as keys in a map (aka complex keys). E.g.
? [ a, b]
: "complex key, scalar value"
Also this is the most banal thing about YAML, something that should be obvious just browsing the examples. And I gave it test suite and link to YAML spec.
Most work in YAML is around super obscure corner cases.
You also get the bonus of nasty bugs like crash bugs and exploitable memory problems become almost entirely a thing of the past. You can ship new software you know is very reliable.
> But, AI is particularly bad at formal methods.
AI? Or GPT? Earlier, reasoning-based attempts at AI seem like they would be a natural fit for formal methods.
By the way, if you spend 90% of time debugging your code, I think that's really sad. Either the programming language sucks, or the codebase sucks, or both. I probably spend 10-20% max of my time on debugging.
The one thing worse than debugging code is debugging "types". Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python.
We can actually try this now. Literally tell the LLM what you want to do and work with it. See how far you can take it. You will of course be asking the LLM for debug line inputs and you will be providing it with outputs as you run the debugger yourself.
Did I learn a bit of java and css and git?- sure, but I was up and running in about 4 hours with a mvp for my 1st one. There is NO way I could "learn" that in that timeframe. I just asked chatGPT 4 how to do it, and it told me. When I didn't know how to commit, it told me (actually I didn't even know the concept). It held my hand every step of the way.
I didn't need to learn something first, I just did it. And I have started doing it at work. "hmm 4 GB of fortinet logs in 20 files of gzip on mac.. how do I find a host name in that? - chatgpt.. oh- 1 line of zgrep.. never heard of it- hey it works.."
admittedly, I am bathed in tech, been hanging around folks talking about projects for years. But NOW I can execute- the problem? When it hits about 500 lines of java- maybe 10 functions, it is too big to drop into the prompt to debug and I don't know enough to fix myself. Solution, make smaller apps, get them working, create data files to reference in json, chain them together. eh, not perfect, but good enough for hobby.
Beware- fools like me who know nothing will be bringing code to production near you soon. Cool that you like to learn stuff, but syntax bores the crud out of me, each to their own, I'm just going to make. I find it more satisfying. Terrifying that code born like mine will end up in someone's prod, but it will.
Before you go on and write such a system it's better to test if the LLM can do debugging to an efficacy level that we require. I don't think anyone has tried this yet and we do know LLMs have certain issues.
But make no mistake, the possibility that an LLM knows how to debug programs is actually quite high. If it can do this: https://www.engraved.blog/building-a-virtual-machine-inside/ it can likely debug a program, but I can't say definitively because I'm too lazy to try.
Though i was surprised i can't easily run pdb instance via a python program, still have to use stdin/out apparently.
Next i want to implement automerge (or semiautomerge) between different outputs which e.g. contain variants of the same function to automatically resolve issues spawned from the model forgetting. That's so annoying
I also suspect a lot of issues are due to the training data being on old SW. I think we can automatically remap this with whitelisted functions and parameters (i recall inspect can do this), blacklisted ones from old version NOT present in the current, and maybe a transformation between the two -- or automatisch regenerate if it's wrong, maybe with a modification to the prompt.
Also talking to it in other languages generates massively different code (i used deepl) so i had the crazy idea of spawning Dockers and just letting this automatic/semiautomatic trouble Shooting+ just parallel generating lots of functions using wildly different inputs (and models) to brute force the problem of having to code
I do need to look into a nice terminal interface for N-way merges and parallel gen monitoring.
The most useful thing for me was making some vim keybinds and scripts to automatically grab Codeblocks, run them and quickly regenerate. You can literally just tell it "DF" and if fixes a pandas issue sometimes
The holy grail will probably be local fine tunes/LoRAs for specific issues or libraries, since it only costs a few $ for one. Sign me up for an expert plotly AI in a box for neat plots please
Edit : i also have literally no idea what I'm doing either, but linting and analyzing generated Code blocks could help expedite this whole process as well. And in principle you don't even have to run it if you know the type is wrong or something.
I don't know what this is called but computer science is ostensibly mathematics so i assume/hope there is some rigor here
_Maybe_ the AI will be able to help with the rewrite, generating test cases and translating business logic, which would be nice.
I also think that the main two reasons behind Alexa/Siri not seeing very high adoption rates are misinterpretation of the voice commands + quality of results. With LLMs, at least the first issue should improve dramatically and if the popularity of ChatGPT is anything to go by, the second issue should also see improvements.
Sometimes it's incorrect or insufficient and it can't handle the more complex solution I'm asking of it, but the code isn't messy and doesn't require me to rewrite it from scratch, just fix what's incorrect.
The only reason why people are so impressed is that chatgpt sometimes gives better results than Google. Which just ought to tell you hiw bad google has gotten.
In my experience, a lot of support requests for bespoke/in-house software go like this:
> User: Why is my wibble being quarked? This shouldn’t be happening!
> Dev: Wibble ID, please?
> User: ID 234567. This is terrible!
> Dev: [rummages in git blame] Well, this wibble is frobnicated, and three years ago [links to Slack thread] you said that all frobnicated wibbles should be automatically quarked.
> User: Yes, but that was before we automated the Acme account. We never frobnicate their wibbles!
> Dev: ...so, is there a way for me to tell if a client wants their wibbles unfrobnicated, or should I hard-code an exception for Acme?
(And then, six months later: “Why are none of Acme’s wibbles being frobnicated automatically?”)
If you could introduce an AI assistant that could answer these questions instantly (instead of starting with a support ticket), it’d cut the feedback loop from hours or days down to seconds, and the users (who are generally pretty smart in their field, even if my frustration is showing above) would have a much better resource for understanding the black box they’ve been given and why it works the way it does.
By having a dialogue, you can resolve only the ambiguities pertinent to the specific question at hand.
People generally didn't seem very interested. Also the bigger problem was that the gpt-3.5 models really wouldn't return the same thing consistently so a minor change could throw other things off.
Also a conversation feels like a much more intuitive way of doing it. So I switched to that.
Of course there will be ambiguity, but you could say anything higher level than assembly is also ambiguous, but that's not usually a problem.
I think it's reasonable to expect that more people will be able to program if this becomes reality. Just as going from assembly to c allowed more people to program.
I wouldn't mind if it had a bit more formalization to the point of math to it for those bits. Was still good enough that I got some usable options from it, though.
Then, down the road, each AI might have a preferred/default language, technology and framework. Quite possibly newly created. This has a parallel to intermediate representations (the earliest of which I'm aware of is p-code in UCSD Pascal). But it is also analogous to compiling C to assembler, and then machine code. Similarly with Java. Wolfram is correct when he talks about needing it to be a representation that the creator can inspect and verify. Not necessarily the least bit easy with a complex project. It will be an engineering journey, but it does spark in me the hope that English (or anyone's native language) becomes the high level language of choice for guiding machines in tasks. Mathematica? Humph. Stephen Wolfram is very pleased with things he's thought of or perhaps synthesized, but I'm gonna say, I think Mathematica is not the generic solution we will want.
So, I think a key challenge is that modern programming languages do 2 things simultaneously.
1. They provide direct instruction to computers.
2. They document precise human intent.
Even if the need for part 1 goes away, part 2 will always be with us. I would expect that if natural language programming becomes a thing, that a dialect forms - akin to legalese - that embodies best practices for precisely documenting human intent.
Sort of like how the SQL syntax seems designed for business admin folks to use, but us programmers ended up using it.
So a bit like `create-react-app` or `dotnet new` or such, but now powered by AI?*
This has existed since at least 1992 (Visual C++) and probably before that (Borland Pascal?).
Not sure if you played with chatGPT, but you can ALREADY ask it to do this and it will give you something that works 50% of the time.
Formalised mathematics are incomprehensible to humans and orders of magnitude longer then anything you can see in textbook or mathematical papers outside automated theorem proving.
Hard disagree. Is Rust similar to natural language? Has English become more similar to <pick the easiest programming language>?
Ambiguity is in my opinion the biggest reason why "plain {insert natural language} programming won't be a thing".. The major challenge in normal commercial software development is not writing working code, it's aligning expectations and identifying assumptions and eliminating ambiguities.. At that point, the coding part is more or less mapping the description to whatever particular words the implementation language is using.
How about Ithkuil? https://news.ycombinator.com/item?id=36022731
Everything in a program is unambiguous, but no useful program stays entirely within its own black box.
Ambiguity still exists between programs. This ambiguity is the source of incompatibility.
Generating code from natural language is closer than you think - https://news.ycombinator.com/item?id=1912530 - Nov 2010 (80 comments)
> I teach computer science and have a particular fondness for introductory CS. The reason Stephen Wolfram is wrong, wrong, wrong about this is that people that have never been taught programming can't express themselves precisely enough in their native language, either; and even among those of us that have been programming for decades, when we express ourselves in natural language we can be very precise but it takes a lot more work and becomes a lot more unwieldy than just writing out our instructions in [pseudo]code.
I used to think the same thing, but I what changed my mind was the noticable increase in precision that came with ChatGPT 4. Before I felt like a monkey - not I actually get things done they way "I intended".
Dang dude, why aren't you a billionaire competing with Google, bing, etc tomorrow?
People are having a discussion about the ability of LLM to generate correct code from imprecise natural language descriptions and how good GPT4 was at that and your rebuke is that it can't do cutting edge algorithm research and an unnecessary ad hominem.
Order update: Your wish is arriving hundreds of years early.
As has already been pointed out, 90% of time is spent on debugging and modifying existing code, not writing new stuff. And of the 10% of coding, 20% is writing the happy path, and 80% is spent in handling errors, corner cases, input validation, and inconsistent domain rules.
Maybe we can have AI tools to help with all this, but there is still a long way to go. And when we get there, it will still take professional developers to use those tools, and to understand all the special cases.
Taking something that's lower level than C and replacing characters with words is not exactly what one'd call a plain English interface, it's just being obnoxiously verbose.
There's a stark difference between trying to somehow half-assedly hardcode this into a language, and having a language that is designed for debugging only, fairly strict in handling corner cases, and then having a natural language interface on top of it so nobody actually has to write the cancer that it likely ends up being.
Rhetorical Algebra
It was developed by ancient Babylonians where the equation was written in the form of words that remained up to the 16th century.
Example: x + 5 = 8, is written as " The thing plus five equal to eight".