- when i declare a variable or function using a suggestive name, it correctly autocompletes what i want 50% of the time and it also takes into account the context; sometimes it just blows my mind how accurate it can be.
- sometimes it doesn't autocomplete even if i delete the line to start over
- i changed the way i code: i rely on Copilot more and more. i start typing my code and i know the prompt will correctly autocomplete my intention, but if it doesn't, it will a couple of lines later. If i stop to think, it goes ahead of me
- it's now faster for me to get an answer from Copilot than searching on Stackoverlow.
Copilot is a great companion, please make it even better! Welcome to the future my friends
Having clever code proposals is surely a big advantage and seems to be a big time saver. But it also means that you have to constantly switch between writing your code and reading/analyzing code that you didn't write yourself, to check if it's correct, if it handles all the cases, etc.
These situations, where we go from one task to another, are typically the kind of situations where we have a very bad perception of the time passed.
It would be interesting to investigate whether this undeniably saves time or not
Often times the comment isn't even needed and just a method name is enough to get it churning out the right code.. I might change a variable name, and some minor stuff but I don't have to search for little things I normally forget ...like framework sugar syntax I've used before but can't remember off my mind but when I see it, I know oh yeah that looks right...
It is definitely the future and it's here to stay and get better with time. I'm super excited!
I'm wondering what it will cost me (read my employer) once it is out of beta.
Considering I am willing to bet that it's gonna be a fair bit.
Denigma is a product that has been around for a while.
Denigma goes beyond a literal line by line explanation, and explains of programming concepts and deduces the business logic and goal of code.
I'm excited to see innovation in the field.
Here's a sample explanation from Denigma:
The code starts by initializing the trampoline. Then it checks to see if there is a passed_info structure in memory, and if so, copies the struct into the passed_info variable. Next, it sets up some variables for use later on:
- info_struct - The smp_information struct that will be used throughout this function
- longmode - Indicates whether or not we are running in 64 bit mode
- lv5 - Indicates whether or not we are running with L1V5 support enabled (if you're unsure what this means google "Intel VT-x" and "L1D cache")
- pagemap - The address of our page table entry array (this is where all of our virtual addresses go)
- x2apic - Indicates whether or not we are using an Intel X2APIC controller
The code is a function that is called when the kernel is booted. The function checks to see if the kernel was booted in long mode or not, and if it was, it sets up a trampoline to call the original kernel.
We're small, privacy-focused, and bootstrapped, and don't have the marketing budget of GitHub, so we'd appreciate you talking about Denigma in your communities.
Nice job.
The hardware requirements (GPU) might be hefty. I'll try to see if I can optimize the size of the model to something smaller.
Very amusing that the explanation goes into great lengths to explain some nuance that doesn't exist ...and gets it completely wrong?
Is this because it was trained on some similar but different code that had this type of nuances explained in comments?
Can I run this on prem.
Can it search though a whole project and tell what I'm doing?
It especially helps when i’m programming in a language or library i’m not familiar with, because i don’t quite understand the syntax but copilot does.
Of course not everybody will use it this way and responsibly used it'll be fine yadda yadda yadda but it is going to shape developer behavior - most especially for developers being a bit lazy (which is all of us sometimes and some of us all of the time).
Also, SLOC implies productivity, which will put a damper on people's reluctance to just add those 35 lines.
Somebody is going to have to dig through all of that code one day looking for obscure bugs. I suspect difficulty in finding bugs is correlated to the square of SLOC (as opposed to writing which is probably closer to linear) so on the other end of the code lifecycle copilot could end up being a masssive false economy.
This could be mitigated by extra vigilant code reviews but I've noticed that vigiliance in code reviews is inversely correlated to PR length (theres a cartoon about this somewhere), so I'm slightly pessimistic it'll stem the boilerplate tsunami.
> "You can’t pay people enough to carefully debug boring boilerplate code. I’ve tried."
I spent at least a half hour trying to figure out what kind of prompt would result in a correct implementation. No matter how explicit I was it kept giving answers that failed to handle the $$P -> $P conversion correctly. Finally I was able to get it to spit out an answer that at least had the right idea, but it had an unrelated bug in it.
The package also has some functions to determine holidays, and I was impressed that Copilot could generate code for new holidays that mimicked my style, docstrings and all. But disappointingly it failed for any slightly obscure holiday (say, Eid al-Fitr), generating code that was just wrong, even when I hinted that it would have to use an alternative calendar library. It kept trying to use the Hebrew calendar for Islamic and Hindu holidays, I guess because I'd already imported it for Hanukkah.
It also seems useless for Spark ETL, having to be handheld to a degree that it would be easier just to write the code myself. My first attempt to play around with Copilot was to see if it could do a salted join (no luck, unless by "salted join" I meant something like "a join on a column called salt").
I'll keep playing around with it, maybe it's just a "me" problem.
This only seems to be able to explain syntax, which seems pretty useless for anyone who isn't new to the language they're using. Am I missing something?
It may also just be an interesting attempt at something that could be great depending on how well it worked - it's an experiment. A general "explain to me what this code does" that was as good as a human explaining overall what is going on could be very useful in more complicated parts of a codebase.
My final thought is that being new to a language and needing to dig around in a codebase is something I do quite a lot and so it could be useful even if it's quite basic.
---
That was also the first time I had heard of denigma.app and their tagline of
We stress-tested it on the worst, most obscure code we could find. That's why we're confident it will work on your complex codebase.
gave me a good chuckle since just trivially modifying their example ... sort of worked
Given:
RUN apt-get update && exit 1 && apt-get install -y libnss3 libgtk-3-0 libx11-xcb1 libxss1 libasound2
- Next, apt-get is used to update the system and exit 1 is used to stop any further commands from running.
- Then libnss3, libgtk-3-0, libx11-xcb1, libxss1, and libasound2 are installed using apt-get.
For good fun, I tried another small edit
RUN exit 1 && apt-get update && apt-get install -y libnss3 libgtk-3-0 libx11-xcb1 libxss1 libasound2
- The code then runs an exit command and uses apt-get to update its package list before installing libnss3, libgtk-3-0, libx11-xcb1, libxss1, and asound2.
It is designed for explaining real-world code in context, and performs poorly on esoteric examples.
Transformer models aren't auto-complete engines. They're semantic graphs describing the probabilities of token positions within a sequence of a few thousand tokens. They go deep, with a nuanced and hierarchical representation of concepts. The representation is functional, conditionally altering the behavior of tokens within its sequence context. Because of the hierarchical nature of the internal rules, when the model "learns" things, it extracts meta-patterns from the training data, and can chain a seemingly arbitrary number of such rules together to produce an output.
Auto-complete systems generally fail at the first level of abstraction. If transformers have sufficient vocabulary size and high quality data, they're capable of learning any degree of abstractions represented within the training corpus.
They're not magic, or intelligent like human brains, but they're allowing programmers to model very complex systems and produce novel material. This ranges from constructed languages to the rules of programming to storytelling and poetry to protein sequencing and more.
You should be in awe of gpt-3, or at the very least, far less dismissive. It represents a quantum leap in the power of software, and we're barely scratching the surface of what's possible. Recurrence and episodal memory, along with Moore's law and/or a substantial decrease in the size of the models, could be the thing that achieves human level agi that lives on consumer grade devices.
And these aren't vanilla "change" commands. It's triggers and check constraints and changing column types and stuff.
def up
execute <<-SQL
alter table users
add constraint active_users_must_have_email
check active = false OR email is not null
NOT VALID
SQL
If I continue and write
execute <<-SQL
a
alter table users
v
validate constraint active_users_must_have_email
SQL
end
def down
execute <<-SQL
ALTER TABLE users
DROP CONSTRAINT IF EXISTS constraint active_users_must_have_email;
SQL
end
What impresses me is that a) it recognized that a NOT VALID index needs to be VALIDATEd and b) it figured out what the down migration was.
copilot? i believe its open to anyone now. this feature? its a nightly build according to the post.
"Create custom prompts
We provide a few preset prompts to get you started: three that explain what a particular block of code does, and another that generates example code for calling a function.
You can customize the prompt and stop sequence of a query in order to come up with new applications that use Codex to interpret code. Creating these can feel like more of an art than a science! Small changes in the formulation of the prompt and stop sequence can produce very different results. The three different “explain” examples showcase strategies that tend to produce useful responses from the model, but this is uncharted territory! We’re excited to see what you use this for."
This isn't for explaining code. That's just an example. This lets us create custom prompts for the OpenAI API using the codex model.
Nevertheless, given Copilot's success so far, hopefully the output will quickly improve.
You can use Cogram by writing ordinary code (and Cogram will complete it), or by writing comments (in which case Cogram completes the code that matches the comment). We've also launched a web app that generates SQL statements from descriptions in simple language.
We're just starting out and would love it if you could try it out, give feedback, and spread the word if you like. It doesn't cost anything -- we have a generous free tier (1000 completions / month).
I'd love to keep using it, but I'm way too familiarized with Sublime Text to change now (and I tried to customize VsCode). I know it's a long shot, but it'd be great if they release an extension for Sublime.