In terms of difficulty, writing code is maybe on average a two out of ten.
On average, maintaining code you wrote recently is probably a three out of ten in terms of difficulty, and maintaining code somebody else wrote or code from a long time ago probably rises to around a five out of ten.
Debugging misbehaving code is probably a seven out of ten or higher.
GitHub Copilot is optimising the part of the process that was already the easiest, and makes the other parts harder because it moves you from the “I wrote this” path to the “somebody else wrote this” path.
Even during the initial write, it changes the writing process from programming (which is easy) to understanding somebody else’s code to ensure that it’s right before accepting the suggestion (which is much less easy). I just don’t understand how this is a net time/energy savings?
It's night and day if you have it enabled or not. There's just no question about the value proposition once you start using it.
I mean, you can tell comments here from people who have actually been using it, and people who have not tried it.
And yet apart from making this very inaccurate comparison you haven't made any argument for why such a thing as Copilot would be useful to anyone. How do you personally find Copilot useful? And why do you think someone whose job demands more than copy/pasting boilerplate code should try Copilot? The onus is on you to convince the skeptics.
It is really fun to see it sort of understand your code though, and every once in a while it does a smart suggestion that could've taken a while for me to figure out myself.
Writing code is easy. Architecture, refactoring, and solving business problems are the hard parts of the job.
Writing new code is also generally the most rewarding aspect of the job. Co-pilot promises to turn that into just another unrewarding chore, like slinging 3rd party libraries together.
e.g. type `pr` in a function with x & t
and it may predict...
The `print(f"The value of 'x' was {x} at time {t:0.00}s")`
Because such code is dumb, tedious and joyless. I have to bite my lip sometimes to convince myself writing them is not a waste of time, because people demand it, but I hate it to my core.
Copilot is that unfortunate boy that has to do all that manual work. It is the ultimate code boilerplate mixer.
It is not going to write all code for you, if you goal is to have it THINK for your then you are due to disappointment. But it would be able to help you be a more efficient and slightly happier programmer.
It never prompted me with any code that was useful. It only ever slowed me down and caused me frustration. It’s nothing like Intellisense. It’s just trash.
He will spend an extra 3-5 seconds using his mouse in order to avoid typing.
Perhaps he is the target market.
now, instead of copying off stackoverflow, it's gonna be off copilot. It will enable a lot more people to code who otherwise would not. Whether this is a good outcome or not...
I could hire 50 juniors that can code tomorrow if I wanted to. But even with an unlimited budget, finding good devs that can make it through a 2 year project without coming out of it with a big ball of unmaintainable shit is difficult.
The gulf from beginner to expert is already big, and the more crutches you use early on, the bigger it's going to get. There's a lot of people that wash out of the industry before they reach the point of being able to comfortably build good software (and be solely responsible for it).
I think copilot is another item in a long list of things that's good for big businesses (who optimise heavily for getting passable results with 1,000 mediocre devs instead of 50 good ones) and terrible for individuals in the long run.
Eventually, I'm seeing another breed of SO questions, making sense of Copilot suggested fragments and seeking reassurance or alternatives... Then possibly the copying off, just as now.
Speaking from my personal experience, I usually write code in TDD style, in which I test the properties of the software I desire upfront, then make it pass with a minimal amount of effort. When I see there's a need for refactoring, I refactor. And I repeat this process until it is done.
The three parts take roughly the same portion of time, and when I'm writing tests, I'm thinking about the functionality and value of the software. When I'm refactoring I'm thinking about the design. When I'm writing the implementation initially, I want it to Just Work™ in the first place, and I find Copilot is great for this matter: why not delegate the boring part to the machine?
> The three parts take roughly the same portion of time, and when I'm writing tests
that bit and have some strong feelings about it. At my current dayjob, writing tests (if it was even done for all code) would easily take anywhere between 50% and 75% of the total development time.
I wish things were easy enough for writing test code not to be a total slog, but sadly there are too many factors in place:
- what should the test class be annotated with and which bits of the Spring context (Java) will get started with it
- i can't test the DB because the tests don't have a local one with 100% automated migrations, nor an in memory one because of the need to use Oracle, so i need to prevent it from ever being called
- that said, the logic that i need to test involves at least 5 to 10 different service calls, which them use another 5 to 20 DB mappers (myBatis) and possibly dozens of different DB calls
- and when i finally figure out what i want to test, the logic for mocking will definitely fail the first time due to Mockito idiosyncrasies
- after that's been resolved, i'll probably need to stub out a whole bunch of fake DB calls, that will return deeply nested data structures
- of course, i still need all of this to make sense, since the DB is full of EAV and OTLT patterns (https://tonyandrews.blogspot.com/2004/10/otlt-and-eav-two-big-design-mistakes.html) as opposed to proper foreign keys (instead you end up with something like target_table and target_table_row_id, except named way worse and not containing a table name but some enum that's stored in the app, so you can't just figure out how everything works without looking through both)
- and once i've finally mocked all of the service calls, DB calls and data initialization, there's also validation logic that does its own service calls which may or may not be the same, thus doubling the work
- of course, the validators are initialized based on reflection and target types, such as EntityValidator being injected, however actually being one of ~100 supported subclasses, which may or may not be the ones you expect due to years of cruft, you can't just do ctrl+click to open the definition, since that opens the superclass not the subclass
- and once all of that works, you have to hope that 95% of the test code that vaguely correseponds to what the application would actually be doing won't fail at any number of points, just so you can do one assertion
That said, it's nice that at least your workflow works out like that!
There are levels of ease of which we have not yet dreamed, especially in the realm of information manipulation.
But I guess we'll find out eventually! And if mine does become the "640k ought to be enough for anybody" quote of this decade, then I suppose there are worse kinds of fame.
OpenAI’s demo from a few months back showed it as a sort of bridge to convert natural language instructions into APIs calls. Eg converting “make all the headings bold” to calls to a word doc api.
Imagine what's possible when that difficulty level shrinks to 0.0001 / 10
Github Copilot is a "code synthesizer"
Xmas time takes me back to one of the most popular "toys" of all time: the Casio SK-1. Music sampler for the masses.
It's like that ;)
If you are writing some non-trivial algorithms or working on some projects which requires delicate handling of things, then Copilot is most likely going to mess up.
But if you are working on many of those frontend code or backend CRUDs which are usually quite repetitive. Then Copilot could be helpful.
At the end of the day is all about trust. Do you trust code you find in SO/Copilot to be good enough for your use case?
In my case I do not trust SO code. Whenever I use SO, if I find some snippet that seems to be the code solution I'm looking for, I copy-paste the snippet on my IDE, read through it carefully, rename variable names as needed, handle edge cases, remove unused code, etc., etc. Any code solution I find in SO gives me the "starting" kick, which is about 10% of the total effort of writing code from scratch. The remaining 90% (to understand the code that is being committed) cannot simply go away. I do not expect Copilot will make much of a difference.
It's not about difficulty, but time. Not the same thing. Easy can still be time consuming.
Have you seen how much time the average developer spends on Stack Overflow and googling for answers?
Just tried something really simple: def is_palindrome
Copilot suggestion was
def is_palindrome(word):
if word == word[::-1]:
return True
else:
return False
So good for technically correct solution but still...
This is an anti-pattern I think in pretty much any language that I know of and something that about half of my beginning students try when they learn about branching..
UPDATE: more howlers along the same vein
def haystack_contains_needle(haystack, needle):
if needle in haystack:
return True
else:
return FalseHowever in Visual Studio Code on a different computer, I got much better idiomatic suggestions.
such as
def is_palindrome(word):
return word == word[::-1]
Like a pulling values from a config dict and initializing a bunch of methods from the class? Or setting up a testcase similar to the one you already have, but with different values? Or cleaning values from a form? It's not a bulk edit. But it's also not thoughtful code-writing. It's monotonous and mundane. And lots of people do a lot of this on a daily basis.
Copilot makes this a cinch.
It is worth mentioning, I suppose, that from Copilot's point of view it is the inverse. Maybe a necessary or at least desirable step towards the inevitable 'Copilot debugger'.
Stuff like:
// Add 100 divs to the DOM in random places
...
// Randomize the color and text of all divs every 1 second
...
I’ve never heard talented programmers say that
# sample call: https://en.wikipedia.org/w/api.php?action=query&format=json&list=geosearch&gscoord=37.7891838%7C-122.4033522&gsradius=10000&gslimit=100
base_url = "https://en.wikipedia.org/w/api.php?"
action = "query" # action=query
format = "json" # or xml
lat = str(latitude.value) # 37.7891838
lon = str(longitude.value) # -122.4033522
gscoord = lat + "%7C" + lon
...
api_path = base_url + "action=" + action + "&format=" + format + ... + "&gscoord=" + gscoord
The old "just copy-paste from Stack Overflow" approach to development is satirised and ridiculed these days (despite being still in common practice I'm certain), because as we all know so well by now, an accepted answer on SO does not always equate to a correct answer. Yes, the SO guys & community do do their best to improve answer quality iteratively (wiki answers, etc.), but there's still a lot of bad answers, and even many of the "good" ones become outdated or don't keep up with modern best-practice (especially when it comes to security).
Omitting urlencoding isn't the biggest crime, but it is a pretty standard URL-building step, and the fact that a tool released this year is spitting out code that omits something so simple is fairly damning. It's also a micro-example of much larger errors Copilot will surely be responsible for. Missing url encoding can be an injection vector in many applications, even if it's not the most common risk, but miss encoding in other string-building ops and you've made your way into the OWASP Top 10.
The big difference between copilot and SO is there's no community engaging in an open & transparent iterative process to improve the quality of answers.
Remember it doesn't actually know an API or how it should be used: it's putting things together to look like typical code. For me that has meant difficult to spot bugs like linking up incorrect variables from the rest of my code.
I wish it could integrate the first SO answer to a generated question, because I always end up there anyway having to fix things.
I've only had it for about a week now but overall I'm happy with it. None of the code I'm writing is crazy cutting-edge stuff and in aggregate I'm sure it saves me more time than takes, including the time I spend reviewing and potentially changing the generated code.
So I would build the query like so:
from urllib.parse import urlencode
urlencode({
"action": "query",
"format": "json",
...
"gscoord": f"{str(latitude.value)}|{str(longitude.value)}",
})
They're the only data you can control, and unless they're strings, it's useless for exploitation. Even denormal floats / INF / NAN won't help achieve an objective.
I broadly agree with you, but people are pummeling Copilot for writing code that I saw hundreds of times. Yes, sometimes I was able to exploit some of that code. But the details matter.
Or did you mean something else?
Oxford English Dictionary, for example, is a human version of defining language and a thesaurus is a completion engine.
Human language didn't suffer by having a dictionary and thesaurus. Computer language doesn't suffer either.
One reason I've championed the development of visual programming (flow-based, node diagrams, etc) is that while you don't want to compress a big complex program down into a single layer and become unreadable in the process, graphical methods are a great way for people to see what options they have available and just point at things they want to try out.
Instead of struggling with syntax at the same time as trying to find out what they can do with a new API, they can engage in trial-and-error to find out the capabilities and what makes it worth using, then build up their competency once they are clear about their objectives.
I'm looking forward to trying this now that it's available for my favorite IDE, but I'll probably want to set up a hotkey to switch it on and off as I need it. Once I get fully comfortable with something I often find code completion tools get in the way of my flow.
let coeff = BigUint::from_str(x).unwrap();
let coeff: <<G1Affine as AffineCurve>::ScalarField as PrimeField>::BigInt =
coeff.try_into().unwrap();
let x: <G1Affine as AffineCurve>::ScalarField = coeff.into();
fn string_int_to_field_el(s: &str)
fn string_int_to_field_el(s: &str) -> <G1Affine as AffineCurve>::ScalarField {
let x: <G1Affine as AffineCurve>::ScalarField = s.parse().unwrap();
x
}
… as a top level comment said, this is optimizing for the wrong problem.
If I want to throw an exception if an object is null or undefined, Co-pilot will do the if and the exception throw using the right error class, and a more meaningful error message that what I usually came up with.
If want to map some random technical data table to something useful in my programming language, I can copy paste the data from the documentation in pdf or html into a comment block, give an example, and co-pilot will write everything in the format I want.
If I want to slightly refactor some code, I can do it once or twice and co-pilot can help me a lot to refactor the remaining.
If I want to have a quick http server in nodejs, I don't have to Google anything.
It's a lot of tiny things like this.
I'm a bit scared for what this means as I don't think being able to faster write boilerplate is something worthwhile. The example ed_elliott_asc made is one of those examples where instead of fixing things so you don't have to repeat yourself, copilot makes it easy to just live with the boilerplate instead.
But do you believe people being slower at writing boilerplate is undesirable?
In a perfect world we’d all have excellent comprehensive metaprogramming facilities in our programming languages and no incidence of RSI (e.g. carpal tunnel syndrome). Code completion is a good tool to deal with these realities.
In the test body, I wrote "foo" to the socket. Copilot immediately filled in the rest of the test: read from the socket, check that the result is "Hello, foo", and fail the test with a descriptive error otherwise.
wtf? How did it figure out that the correct result was "Hello, foo"? I changed "Hello" to "Flarbnax", and sure enough, Copilot suggested "Flarbnax, foo". I added a "!" to the end, and it suggested that too. After pushing a bit further, the suggestions started to lose accuracy: it would check for "???" instead of "??" for example, or it would check for a newline where there was none. But overall I came away very impressed.
The thing I find really good is it can predict what I will do next. Say if I have a list of columns in some text somewhere in the project when I write one “df = df.withColumn(“OneItemInList”)”
Copilot will then add the same for all the other items - is really nice
It sort of feels like pair programming with an undergraduate, except copilot never learns. That isn't to say it's bad, more that it is just a tool you can hand simple stuff off to, except the handoff is zero-effort.
EDIT: I will say that there are times when it makes up fantasy constants or variable names, that seem plausible but don't exist. An eventual version of Copilot that includes more normal autocompletion information, so it only suggests symbols that exist, will be a stronger tool.
There's the benefits that a lot of people mentioned, but to me the biggest benefit is I can avoid procrastination. Usually when I'm blocked on something I'll run a search in the browser, but very quickly I end up going off the trail and just browsing the web and losing a lot of time. Now when I'm blocked I simply type the comment of what I'm trying to do and the autocomplete suggestion is pretty damn good and unblocks me very quickly. More surprising of all, it somehow understands my code style by looking at the context.
I was able to write {"Settings":...} and Copilot completed with {"Settings": "Configurações"} that tool is simply amazing.
More verbose languages like C++ become less obnoxious to write in. I know RSI has been mentioned and any tool which cuts down on excessive typing will help with that.
It sometimes reveals bits of the standard library I wasn't aware of in unfamiliar languages. I can write my intent as a comment and then it may pull out a one-liner to replace what I would have normally done using a for loop.
The main downside I've observed is that if I'm not trying to reign it in, it can result in a lot of WET code since it can pattern match other areas of the surrounding code but can't actually rewrite anything that has already been written. It is important to go back and refactor the stuff it produces to avoid this.
For example, I didn't know about self.fail() in unittests and had never used it, but Copilot suggested it and it produced the most readable version of the unit test
It's very good at translating between programming languages, including pseudocode.
It can write a lot more valid code much quicker than any human, and in a whole slew of languages.
I haven't had the urge to use it much after playing around with it constantly for a few days, but it was pretty mind-blowing.
It‘s useful enough for me, as a magic autocomplete.
https://twitter.com/ridiculous_fish/status/14527512360594513...
In a way, it does the dirty pipes surprisingly well. But when it comes to implement the core of the algorithm, it is not there yet, but the potential is huge.
Also, it’s like a more clever auto complete most of the time, even when it’s wrong in calling a function you can use it as foundation code to go faster.
And you don’t need to think too much about it, it really keeps you in the flow.
So for example I was working with processing an image to extract features and a few variants of docstrings for the method got me a pretty close to working function which converted the image to gray scale, detected edges, and computed the result I wanted.
The helpful thing here was that there were certain APIs that were useful as a part of doing this that it knew but which I would have to do look up. I had to go through and modify the proposed solution: it got the conditional in the right place, but I wanted a broader classification so switched from a (255, 255, 255) check to a nearBlack(pixel) function which it then autocompleted successfully. I also had to modify the cropping.
When doing a similar task in the past I spent a lot more time on it, because I went down a route in which I was doing color classification based on the k nearest neighbors. Later I found that the AI I was working on was learning to exploit the opacity of the section of the screen I was extracting a feature from in order to maximize its reward, because it kept finding edge cases in the color classifier. I ended up switching to a different color space to make color difference distance functions more meaningful, but it wasn't good enough to beat the RL agent that was trying to exploit mistakes in the classifier.
Anyway, what I'm getting at here is that it is pretty easy to spend a lot of time doing similar things to what I'm doing and not get a great solution at the end. In this case though it only took a few minutes to get a working solution. CoPilot didn't code the solution for me, but it helped me get the coding done faster because it knew the APIs and the basic structure of what I needed to do. To be clear, its solutions were all broken in a ton of ways, but it didn't matter it still saved me time.
To give another example let's say you have a keyboard key event press and you weren't sure about how to translate that into the key that was pressed. key.char? key.key? str(key)? key.symbol? A former method of figuring out what the right key might be is looking up the code, but with CoPilot you type '# Get the key associated with the key press' then hit tab and it gives you code that is broken but looks perfect and you gain a false sense of confidence that you actually know the API. You later realize after being amazed that it knew the API so well that you didn't have to look it up that actually the key press event handles symbols differently and so it errors on anything that is used as a modifier key.
My general impression is something like: Wow, this is amazing. It understood exactly what I wanted and it knew the APIs and coded up the solution... Wait, no. Hold on a second. This and that are wrong.
Is it irrational that this makes me a little anxious about job security over the longterm? Idk why but this was my initial reaction when learning about this.
Given the scenario where copilot and its likes becomes used in a widespread manner. Can it be argued that this might improve productivity but stifle innovation?
Im pretty early in my career but the rate things are capable of changing soon doesn’t sit too well with me.
In the 50s, we programmed computers with punch cards. Who does now? How many web developers today could tell the difference between `malloc` and `calloc`? Probably not that many.
For a lot of developers, programming today bears very little relation to programming decades ago. Copilot is like any other innovation - it obsoletes some skills, and it introduces new ones.
I doubt copilot will reduce the need for engineers: but it may change the work they do. But that's no different to any other industry.
It would be more like we still write asm but we have editors that let you write a little C code and then it spits out a paragraph of ‘reasonable’ asm that still has to be maintained.
Every time this happens, everyone just shifts the goal posts and they now want more features, faster. The majority of software out there sucks. If programmers are now 2x faster, users will demand that some random crud app be at Google software quality. And Google's software will be unimaginable by today's standards.
All of this will increase the value delivered by software, which will bring in greater revenue, which will be reinvested in more developers.
- Copilot is qualitatively different from the kinds of automation of programming we've seen before.
- It's barely version 1.0 of this kind of thing. Deep learning has been advancing incredibly for a decade and doesn't seem to be slowing down. Researchers also work on things like mathematical problem-solving, which would tie in to "real work" and not just the boilerplate.
- In past examples of AI going from subhuman to superhuman, e.g. chess and go, the practitioners did not expect to be overtaken just a few years after it had not even felt like real competition. You'd read quotes from them about the ineffability of human thought.
What to do personally, I don't know. Stay flexible?
I could see it being huge for the GUI scraping market. Or imagine a browser plugin that watches what you're doing and then offers to 'rewire' the web page to match your habits.
Imagine some sort of Clippy assistant watching where you hover attention as you read HN for example. After a while it says 'say, you seem to be more interested in the time series than the tree structure, would you rather just look at the activity on this thread than the semantics?' Or perhaps you import/include a sentiment analysis library, and it just asks you whether you want the big picture for the whole discussion, or whether you'd rather drill down to the thread/post/sentence level.
I notice all the examples I'm thinking of revolve around a simple pattern of someone asking 'Computer, here is a Thing; what can you tell me about it?' and the computer offering simplified schematic representations that allow the person to home in on features of interest and either do things with them or posit relationships to other features. This will probably set off all sorts of arms races, eg security people will want to misdirect AI-assisted intruders, marketers will probably want tos tart rendering pages as flat graphics to maintain brand differentiation and engagement vs a 'clean web' movement that wants to get rid of visual cruft and emphasize the basic underlying similarities.
It will lead to quite bitter arguments about how things should be; you'll have self-appointed champions of aesthetics saying that AI is decomposing the rich variety of human creativity into some sort of borg-like formalism that's a reflection of its autistic creators, and information liberators accusing the first group of being obscurantist tyrants trying to profit off making everything more difficult and confusing than it needs to be.
Noone knows what future holds, so some anxiety is just a fuel for adaptation.
For example, should Copilot take a widespread use, the number and scale of projects that will have to be maintained expands too. Moreover, making sense of the quilt patchwork of bits and pieces of code, I'd guess, written in many iterations/versions of Copilot's prowess is a very much a secure, if soul-killing, job for many. Not much different from what maintenance jobs are now. You're lucky when a project retains some clear overspanning architecture/style.
Anybody remembers the joys of GUI wizards, with tons of auto-generated code that "just works, just now"? Remember that desire to suggest a healthy rewrite? Well, now you could probably also promise that it would be an even quicker rewrite too!
But even then, the final responsibility for the code is on the programmer. One maybe could forge the code quicker, but code review is still supposed to be a human's job. Hopefully.
Future developers might be more like architects guiding AIs and then occasionally jumping in to hand-hold or correct the result.
I've been programming since the '80s. It's my opinion that the age of humans writing code is coming to a close. Perhaps another 20 years or so, with the peak in ~10; but I'm less certain about the timeline than the destination. There will still be a long tail, but most of the human work will shift to design and wrangling algorithms. The remnant will be hobbyist, such as Commodore 64 programmers today.
I expect most human intellectual activities of today (from coding to scriptwriting to medicine) can be performed by machines if the current trend continues.
One of the main value props for writing meaningful unit tests, is it helps the developer think differently about the code he is writing tests for, and that improves quality of the code composition.
* It promotes actually looking at the code before considering it done
* It promotes refactoring
* It helps to prevent breaking changes for stuff that wasn't supposed to change
To me, the value of copilot helping to write tests is that we, the engineers, come up with the test cases, and copilot helps write the code for that case.
I think humans will still be more imaginative in the test cases they can dream up (although I’ve never used an automatic generator, maybe they’re better than I think), but almost all test code is boilerplate, either in the setup or the assertions.
If I don’t have to write that repetitious, yet slightly different boilerplate for each test case, that frees me up to design other interesting test cases (as opposed to getting tired of the activity by the time I cover the happy path) or move on to the next bug/feature work.
If big tech was trustworthy I would use this with glee. But when I see how the world is turning, I'll continue to type boilerplate by hand (as long I'm allowed).
Be sure you don't commit your code to github...as that's literally uploading your code to Microsoft servers, I mean if you just don't trust them for being 'Microsoft'.
Also if this feature would be paid tomorrow, I think I would pay for it. It’s really noticeable when I don’t have copilot enabled now.
Oh and, autocompletion doesn’t work with markdown files because of markdown plugins I think? But this is another level of insanity: when I’m writing english it figures a lot of the sentences I want to write. Makes me question if I’m just a deterministic individual with no choice.
Delegating the implementation of something that you are notoriously never supposed to roll your own, to a text generator AI.... What could possibly go wrong?
First, as much as I don't like the idea of Copilot, it seems to be good for boilerplate code. However, the fact that boilerplate code exists is not because of some natural limitation of code; it exists because our programming languages are subpar at making good abstractions.
Here's an example: in Go, there is a lot of `if err == nil` error-handling boilerplate. Rust decided to make a better abstraction and shortened it to `?`.
(I could have gotten details wrong, but I think the point still stands.)
So I think a better way to solve the problem that Copilot solves is with better programming languages that help us have better abstractions.
Second, I personally think the legal justifications for Copilot are dubious at best and downright deception at worst, to say nothing of the ramifications of it. I wrote a whitepaper about the ramifications and refuting the justifications. [1]
(Note: the whitepaper was written quickly, to hit a deadline, so it's not the best. Intro blog post at [2].)
I'm also working on licenses to clarify the legal arguments against Copilot. [3]
I also hope that one of them [4] is a better license than the AGPL, without the virality and applicable to more cases.
Edit: Do NOT use any of those licenses yet! I have not had a lawyer check and fix them. I plan to do so soon.
[1]: https://gavinhoward.com/uploads/copilot.pdf
[2]: https://gavinhoward.com/2021/10/my-whitepaper-about-github-c...
The fast inverse square root algorithm referenced here didn't originate from Quake and is in hundreds of repositories - many with permissive licenses like WTFPL and many including the same comments. It's not really a large amount of material, either.
GitHub claims they haven't found any "recitations" that appeared fewer than 10 times in the training data. That doesn't mean it's a completely solved issue though, since some code may be in many repositories yet always under non-permissive licenses.
> and I would argue that it will not be the case for ML models in general because all ML models like Copilot will keep suggesting output as long as you ask for it. There is no limit to how much output someone can request. In other words, it is trivial to make such models output a substantial portion of the source code they were trained on.
With the exceptions mentioned above, what you get back from asking for more code won't just be more and more of a particular work. Realistically I think you'd be able to get significantly more from Google Books.
With the exact same comments?
> many with permissive licenses like WTFPL
So it would be perfectly legal to do whatever I wanted with the source for GCC as long as there was a single fork on github that replaced the GPL with a MIT license? Quite sure the FSF would be perfectly fine with that.
Where did it come from then? And what license did the original have?
> and is in hundreds of repositories - many with permissive licenses like WTFPL and many including the same comments.
If the original was GPL or proprietary, then all of this copies with different licenses are violating the license of the original. Just because it exists everywhere does not mean Copilot can use it without violating the original license.
> It's not really a large amount of material, either.
No, but I would argue that it is enough for copyright because it is original.
> GitHub claims they haven't found any "recitations" that appeared fewer than 10 times in the training data.
Key word is "claim". We can test that claim. Or rather, you can, if you have access to Copilot, you can try the test I suggested at https://news.ycombinator.com/item?id=28018816 . Let me know the result. Even better, try it with:
// Computes the index of them item.
map_index(
> With the exceptions mentioned above, what you get back from asking for more code won't just be more and more of a particular work. Realistically I think you'd be able to get significantly more from Google Books.
That can only be tested with time. Or with the test I gave above.
I think that with time, more and more examples will appear until it is clear that Copilot is a problem.
Nevertheless, a court somewhere (I think South Africa) recently ruled that an AI cannot be an inventor. If an AI cannot be an inventor, why can it hold copyright? And if it can't hold copyright, I argue it's infringing.
Again, only time will tell which of us is correct according to the courts, but I intend to demonstrate to them that I am.
By definition, boilerplate is code that is repetitive. That repetition should be minimized by the programming language. If there are assumptions and guesses, that's because there's less repetition than you think.
I am talking about eliminating repetition, not code with differing assumptions.
Disgusting.
Copilot sometimes reproduces code verbatim. You can't use open source code except under the terms of the license. Authors whose code may be reproduced by Copilot need to grant a license to downstream consumers, and republishers of Copilot-generated code need to adhere to the terms of that license.
Copilot is inserting ticking time-bombs into its users' codebases.
Which I think was fairly predictable.
What wasn't predictable was that someone would ship this Copilot anyway, consequently exposing their company and their users' companies to liability.
Imagine if you hired an intern who was copy&pasting bits of GPL'd code throughout your system. This would not be a good job, it would be something that needed immediate attention from legal counsel and others, and mean reverting every commit the intern from heck made if you couldn't prove convincingly it wasn't tainted. Especially if you're a startup, who needs to assure investor due diligence in good faith that you actually own the IP.
Letting your intern blindly commit to your code base seems like the bigger issue here. The entire purpose of an internship is to learn and to be guided by professionals, not to be treated as a cheap laborer. You don't hire interns, you train interns.
Have you used copilot or are you speculating?
(And people couldn't tell in code reviews, nor on other occasions to see the code, since it's not normal to recognize GPL'd code on sight; everyone just assumed the developer was productive.)
This has not been my experience. I was dropped into the developer team and expected to know the entire tech stack and was not trained by anyone from the company at any point. Have I been bamboozled??
A dependent type is a type that depends on a value. What something is depends on something else; What something is is derived from something else – with computation. What dependent types are are calculations of type structure.
Unbounded, this becomes intractable like all unbounded computational expressivity. So that’s what modern dependent type systems need to solve—and they do. There exist formal results that frame the computational expressivity of dependent type systems and make perfectly feasible and tractable the task of deriving significantly complex types. Automatically.
The Idris language is a great example of where we’re heading.
The reason I’m more interested in this sort of thing rather than Copilot is that dependent type systems are based on formally rigorous methods. You end up with formally verified programs, by way of the same mechanisms that allow you to derive them automatically. They’re also easier to compose, for the same reasons.
Edwin Brady’s Idris demonstratons on Youtube show a bit of what’s possible. In one, the compiler automatically writes a formally-correct type-directed matrix multiplication function. In another, run-length encoding and decoding functions are generated from a type definition.
The book Type-Driven Development with Idris is a great read. Mind-expanding.
Programming is automation. The automation of automation is… dependent types.
I think the reasons are completely pragmatic and boil down to two things.
The first is that rigorous systems usually take longer to build and meanwhile the slop ships first and gains mindshare.
The second is that rigorous systems are usually built by academics or specialists in some specific vertical and lack the easy installation, easy onboarding, and integration with other platforms that more pragmatic sloppy platforms tend to prioritize first. Those things get prioritized first because sloppy languages and systems tend to evolve from makeshift "shop jigs" used to get stuff done, not from research.
Very eager to see Github Copilot catchup to some bright line of signal v noise.
And the worst thing about that is that you don't get the context of the Stack Overflow threads, where people discuss the impact of the given solution and alternatives. So after a week, off it went for me.
For those who don't know, he's essentially the godfather of vim plugins - I even have an entire 'tpope section' in my init.vim
- less than perfect import/types/var suggestions that LSP in typed languages would've made perfect suggestions for (e.g named import in go would use the package name instead).
- latency feels a bit high and my thoughts would get interrupted waiting for a suggestion to come.
For the former, I wonder hard feasible it would be to give structured suggestions to the LSPs that it would swap for correct var names and imports and such. Or test each suggestion with the LSP for error counts and offer the least erroring suggestion.
How high? That's interesting.
<!-- Details on the fall of rome -->
<p>The fall of rome was a great event for the people of rome.There tends to be a lot of repetitive code in the world. I primarily write JS, Py, and Rust. Sometimes, I might declare something like a function table, and Copilot will automatically fill in the class definition with everything I defined.
I'm not using Copilot to write new algorithms or solve library-specific problems, but it sure is next-level in picking up patterns in a file and predicting where you want to go next. Obviously, good code is succinct (not repetitive), but it sure is helpful when in that early prototyping stage. I admire it's ability to infer a correct assertion when writing Unit Tests - it made it much easier for me to write tests recently and helped me recognize a few bugs.
And I do a lot of PR approval, and I shudder to think about PRs stitched together in haste because look how easy it is now to crank out code!
And lastely it doesn't work well when you have a large legacy codebase that your working within.
I added an issue. https://github.com/github/feedback/discussions/6847
Anyone else install in neovim?
https://plugins.jetbrains.com/plugin/17718-github-copilot/re...
However, I imagine a lot of people signed up just before me, so I was probably at the end of a long list. The wait wouldn't be this long if you sign up today, I reckon. I'm just guessing though.
I didn't allow it since I think it implied uploading my code to their servers.
Ultimately I'd like to have a conversation with a machine, where I can describe what I want, and the machine can serve me a possible answer, and then I can respond to clarify the idea.
Currently many programmers do not take the time to really understand how/why their code works -- programming without understanding 1.0. Essentially make library calls and fuzz around with the arguments until it appears to work. [Not wanting/suggesting to go back to the world before libraries/code completion, just stating where we are now.]
This will enable programming without understanding 2.0 -- not only will you not know how/why a particular function call works, you will now fail to understand why you want a sequence of functions in a particular order.
My take is it's what you make of it. Copilot is only equivalent to copy-and-pasting from stack overflow if that's how you choose to field its suggestions.
As an example, I've enjoyed typing "const one_day_in_ms" and letting it finish it out with "1000 * 24 * 60 * 60". I already knew how to do that, but having GCP finish it for me and verifying on my own didn't make me feel stupider, it made me more efficient. I have more interesting problems to tackle.
On the other hand, another coder could have not known this calculation and thrown their trust into GCP. That's bad practice and it's on them, not on the tool.
Sometimes GCP gives me code that it learned from bad coding patterns. I know how GCP works and I know to look out for that, so I ignore those suggestions.
Of course, sometimes I don't know if what looks like a good idea from GCP is actually not. I take that on as my responsibility to trust but verify. If it's writing some function to slugify a string for a URL, I check it against what people are discussing online. Does it defeat the purpose of GCP in this case if I have to check it on my own? Probably, but it's only in these specific instances when I'm doing something I'm not familiar with.
> Read and agree to the GitHub Copilot additional telemetry terms.
Can anybody comment on the privacy aspects of this? Is the telemetry reasonable? Why on earth do I need to login: presumably so that they can associate my coding with my account to structure the data they are gathering?
This is like social media (or even the Internet as a whole) and say, our parents' generation. Countless times I receive links to Facebook posts or random articles that somebody thinks must be true, simply because The Oracle (i.e. their smartphone) showed it to them. For much of the older generation, smartphones are these all-knowing repositories of wisdom, and anything they come across while using them is likely to be true. This is why I think misinformation has spread so easily.
I imagine Copilot going down a similar path. The next generation of programmers who didn't grow up with knowing how to sift through API docs or SO answers for the right bit of code, or who's attention spans have been fizzled away, will love the idea of Copilot. Instant gratification in the firm of a tool that can seemingly do your work for you. This will be dire consequences for their ability to code and think for themselves.
I filed a PR because it was a bit frustrating to go through the entire setup and then find out I needed to be granted access.
The real problem is indeed the code quality, since Copilot does actively provide low-quality codes, and this will bite a lot of people. I guarantee. I don't think this massive learning approach can't be the solution. We need an extra kick.
My shower-thought solution for this garbage-spewing problem is to design a new library with Copilot in mind. Tighten up the interface, and use strict patterns instead of domain-specific hacks.
In other words, we can make libraries so lame that Copilot (and newb programmers) simply can't produce low-quality code. Just disregard smart programmers with fast hands. They don't need any help anyway. Don't even try to target them, because it's gonna cause more stupid flame wars...
Looks like for VSCode the shortcut on Linux is Alt-], see: https://github.com/github/copilot-docs/blob/main/docs/visual...
But for neovim, it doesn't mention anything about it in the docs: https://github.com/github/copilot.vim/blob/release/doc/copil...
And, nothing happens when pressing Alt-].
Maybe I'm missing something obvious, but I feel like CoPilot should have the ability to allow a user to define a 'class template' in the form of a block comment and then allow a user to write "make this class" or something.
*Big being it's probably saved me around two hours of manually typing out 'public property... etc.'
- Co-pilot will not replace software engineers, however ...
- I do think it will in some cases help them be more productive as some have expressed in this thread already.
- Once they open it up to fine-tuning on your own codebase I suspect it can be used to bring new engineers upto speed faster plus it get will get more trustworthy.
- I do have concerns about the legal aspect of selling software built with assistance if co-pilot but a lawyer could probably get me comfortable (or not!)
- I have personally found it useful for data science type tasks especially getting familiar with new libraries.
overall, a new forms generator with a somewhat terrifying amount of horsepower.. zero trust of microsoft here, basically
The plebs should also play a little divide and conquer now and then.
it's said because vim lacks 'virtualtext' feature.