I know this first hand, building a developer tool startup and failing to reach any level of revenue. In the end, the tech was bought out by a larger company to recover a fraction of our VC investment.
The challenge is that when you're building software for developers, they already know how it must work.
It's like trying to sell magic tricks to magicians. Sell magic to regular people, and you'll see some significant revenue.
I've used Kite before. It was ok. But I am a SWE. It's entirely possible that Kite would have seen major adoption if the push was towards non-technical folks trying to get their feet wet in software. Eg: Data scientists or business.
The reason why BI tools sell so well at the moment is that you have tons of C-level execs that like the appeal of a business-optimizing tool requiring little to none of any actual software development.
Let that be a lesson to everyone. You can't blow away developers. They're just too damn ~~smart~~ well-informed.
Edit: Another anecdote: A buddy of mine built a bespoke OCR and document indexing/search tool. He has ~60 paying clients (almost exclusively law-firms and banks) that primarily work with printed pages on paper. No Saas. No free tier. The client data resides on an on-premise Windows box, avoiding issues with sensitive data in the cloud etc.
He's a solo dev with support contracts and nets something like $1000/month from each client.
For your average lawyer/paralegal, the ability to locate and reference a single page from thousands of pages in under a second is magic. So they pay for it wholeheartedly.
I see it like a construction worker that uses his own tools instead of the broken down ones of the site. It makes good business sense that I can work faster and better when I use better tools. I also pay for other tools such as dbForge and SmartSvn/Git but Jetbrains' tools have been the longest running ones (I hope this does not sound like I am a fanboy or something).
When GitHub Copilot decided they wanted to charge $10 a month after the beta was over I noped out.
For what it does, it sure as shit isn't worth paying more than my yearly subscription to IntelliJ...
If you value your time just even a little bit, consider how many of those tools are multipliers.
Obviously not 'JIRA' for a single dev, but in many cases JetBrains is worth it.
Would you wear Basketabll sneakers out for a jog? Well you could, but if you're going to run buy a pair of running shoes. Probably once a year. Costs about the same as Jetbrains for a year - as a very crude analogy.
At least in some cases.
Would be silly for corpo to not buy it for developers
Sublime Text.
I sat through scores of interviews and pairing sessions with developers back when Sublime was a thing and the vast majority (>90%) of devs would rather exit out of that pop-up asking for support then pay the measly $30 or whatever regardless of their massive incomes and increased productivity that Sublime brought them.
We developers are no more altruistic than anyone else regardless of the lies we fed ourselves in the early days of FOSS, internet, bitcoin, etc.
:(
Still a relative drop in the bucket for how powerful ST is and how much value its users derive from it, but there _is_ a bit of sticker shock when comparing it to most other software.
I say this as someone who, as a broke college student, got very good at hitting esc every 10 times I saved (which is how often it asks). I eventually switched to VSCode, and the rest is history.
Converting just 5% of users to paying customers is considered pretty good for shareware/freemium, so just because you saw a lot of people using the trial does not mean it is unsustainable. I have no idea how much the developer of Sublime Text makes, but considering that they have been around since 2008 I would assume it's definitely sustainable.
- time
- headache
- improve my quality or quantitive results
Software developers very simply would rather build their own half assed solution to a problem rather than pay for a half assed solution.
Offer quality, we'll pay.
The other thing is they have to be tools I want to use. I am an outlier I am sure. I hear often "let your employer pay for it" but they don't always necessarily pay for the tools I need to use. Having my own JetBrains license grants me strong freedom.
Jetbrains' pricing model is also good, they reduce price each year (until 3rd), so you get rewarded for having a long term subscription. If you break commitment you get the regular pricing and you start over.
I remember trying Kite, but I removed it once I saw the pricing. It was more expensive than Jetbrains IDEs (which are less than 2$ a month for individuals when you pay in a bundle - 149$ at the time) which bring much more value for the money. For me it didn't make sense to pay 20$ for just incremental improvement (if even that) over Jetbrains Intelisense.
Hijacking the quote...
I can't count the number of times I see well-paid developers using the Sublime Text trial.
A bit tangential to the original post, but where does this belief that data scientists are non-technical folks? I am a data scientist myself, and in my view it's way more technical than most software development. Albeit I wouldn't still call neither data scientists nor software engineers "smarter" than the average.
Sure, if you want to train your bread and butter text classifier it just takes 10 lines of boilerplate code. But you don't need an AI-assisted tool for that - you just go to hugging face, copy paste those 10 lines, done, it's certainly faster than getting some AI-assisted code editor work for you.
For everything that is a bit more complicated, you need endless adjustments to your code, and it's quite unlikely more than a handful of people before you ever wrote the same code. It is, indeed, a somewhat painful and slow process (because just "testing" your code often takes minutes, if not hours, so finding out bugs becomes annoying). And a somewhat simple, AI-based, error highlight tool might be useful to weed out the most stupid ones and save some time.
But I will never trust something like copilot (or Kite, I guess, which I never tried) to write my code for me, as the challenging parts of the work involve long-term connection between different pieces of code (data loader, loss function, model function) that are written independently but must "cooperate" in a very non-trivial way. It is not at all uncommon that I make hours-long screen sharing calls with a colleague, discussing non-trivial mathematical computations, only to end up changing one or two lines of code that don't have an immediate link with the problem we are trying to solve.
This kind of things are notoriously hard for AI to grasp, so they can't do any decent job in writing that for me. Add on top that a lot of the code you find freely online is just ridiculously bad or broken, and you might only get unusable models generated by AI engines trained on those.
So, what kind of work are you referring to when talking about "data scientists or business" in your comment?
In the software world people seem to be referred to as technical when they write software systems not as much as singular scripts.
Data science is definitely technical but a lot of code work tends to happen in Jupyter notebooks or something similar. The main challenge is in understanding the ML/AI algorithms, the possible choices for your analyses that actually make sense for the problem, ... .
Besides that, due to the AI hype, there are so many people in data science who don't know much about coding or software engineering. Therefore, helping these people might be profitable (or not).
I can't speak very broadly, but at least for my company most data scientists are not doing the kind of work you describe. There certainly are some folks constructing and training complex machine learning models, but I think the majority work on the level of more basic statistical models and rules of thumb, where a project's final output might be a dashboard or presentation. Arguably some might refer to this as data analysis rather than data science, but none of these terms are particularly well defined.
That's not to say they aren't technical in some sense. All of them can and do code to one degree or another (with perhaps the exception of a small number of people who've been in the industry for decades), though not all of them do so with high proficiency or attention to software engineering best practices. That also goes for some of the engineers where I work, admittedly.
All in all, the broad level of technical aptitude has grown over the past few years. But not everyone with the title of data scientist is a machine learning specialist, nor are they necessarily skilled at software engineering.
Edit: As for Copilot, I found it worse than useless. It miserably failed every test I threw at it, from machine learning to (especially) Spark data pipelines, only redeeming itself with a string handling problem -- for which the solution was still entirely wrong but at least interestingly wrong. I frankly don't see how anyone pays for it, though perhaps it's better for projects with a ton of boilerplate.
> I know this first hand, building a developer tool startup and failing to reach any level of revenue.
Just because your startup failed doesn't mean an entire category is unsustainable.
I've been living from sales of a developer tool for the last 10 years, and there are plenty of other paid developer tools out there that show developers absolutely do pay for developer tools.
Now, maybe some of the startups have unrealistic expectations. A Python documentation reader probably wont turn into a billion dollar revenue company no matter how smart it is.
But I'm pretty sure there is a market for dev tools. Maybe the market is smaller, or harder to crack than you thought, but saying "there is no way" isn't going to help anyone.
I believe this is incorrect. I pay for many tools, but I would not pay for Kite. The problem is not that developers don't pay for tools, but that Kite, or AI-assisted code, does not address a pain point. It's a slight improvement, but I don't feel pain when I need to write code without it.
That's different from something like CI tools that I pay for. When I need to wait long for CI to finish I get annoyed. That's when I pay.
I tried Kite a few years ago but didn't feel like I was getting much value out of it and never payed for it.
In contrast, I started paying for Github Copilot as soon as it was no longer free.
¯\_(ツ)_/¯
Just FYI, magic tricks are basically only sold to magicians. There's a thriving market of magic shops, especially online, where magicians go to buy new tricks (in the form of books/videos), new "gimmicks", etc.
I'd wager that a significant portion of all magic that is done is actually by magicians, for magicians, and partly in order to sell magic tricks.
I think it's just domain awareness, and the "they're smart" trope needs to be dismantled.
I think it plays into the technocracy problems we have now. We can solve it, we need more tech. More more more. People think we can solve social/political problems with tech - insidious.
I'm sorry it hasn't worked out for them, but they get my respect for this unusually frank self-assessment, real humility, and following through on the fine words with the actions of sharing their tools they built. They achieved a lot and I hope their future endeavors are wildly successful.
Do you mean that what you built didn't worked as it should? I don't understand, I've paid multiple times for tools that I find useful, even if they weren't perfect.
This misconception has been promoted by companies with an interest in promoting their platforms, using the expeditive procedure of subsidizing (often inferior) tools, with the collateral effect of making impossible for tools vendors to compete.
But by no means it's a law of physics. Make something programmers want. It's weird how little have the tools improved in twenty years.
That was a really shocking insight for me. We do not own our means of production. And suffer all the textbook consequences that follow.
Maybe unions could help with that. Imagine using union-funded licenses, compute, storage etc. to experiment with your side projects, build your prototypes without risk of losing IP to your current employer.
However, I would never pay for stuff that I can get free. I could talk to my company to buy it, but I would settle for something close and free if it come to that.
I am an individual developer and I pay for my PHPStorm and GitHub CoPilot - it saves me so much time, I could never imagine. This software is completely worthy to be paid for, even considering "free" copies are available.
The blog post was quite clear on the ”the tech doesn’t work” part, which seems like a more likely reason for their demise. Selling developer tools is hard, but selling non-functional tools is exponentially harder
I never used Kite, but I've tried Github Copilot twice, and found it marginal at best (and distracting at worst - which is why I turned it off both times). If Kite was similar, the reason I'm not paying is that coder AIs are not providing any value.
Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.
Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.
The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.
My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
This, so much. My code since using Copilot is easily ten times better tested than it was before, and I wasn't especially lazy when it comes to testing.
Given 1-2 hand-written unit tests, Copilot can start filling in test bodies that correctly test what's described in the function name. When I can't think of any more edge cases, I'll go prompt it with one more @Test annotation (or equivalent in another language) and it will frequently come up with edge cases that I didn't even think of and write a test that tests that edge case.
(One great part about this use case for those who are a little antsy about the copyright question is that you can be pretty darn confident that you're not running a risk of accidental copyright violation. I write the actual business logic by hand, which means copilot is generating tests that only interact with an API that I wrote.)
Testimonials of this form are near worthless to a company. Maybe it's true for you. Statistically, it's highly likely to be misleading.
People overestimate their willingness to pay for something for a number of reasons, but one of the biggest is that they incorrectly visualise what the choice to pay or not looks like. They often imagine a moment of abstract choice after which everything remains exactly the same but some small amount of money magically vanishes from their bank account. In reality, paying for something is a tedious inconvenience, and not paying for it more often takes the form of never getting round to putting your card details in than consciously deciding "this isn't worth it".
It can be taken to questionable extremes, but there's truth in the idea that the only real evidence as to what customers will do is what they actually do, not what they say they will do. I don't know if their interpretation is correct, but it sounds like Kite at least has evidence of the former sort.
(Don't have much experience with it)
Since we literally build all of this our B.S. detection meter is really high.
Kite thought it can go after the up and coming new developers by doing slightly shady things.
However, developers also have an incredible allergy to such tactics and it forever taints your brand.
So overall, developers do pay for tools, just not useless ones with shady growth tactics.
I briefly tried Kite a few years ago. I didn't notice anything shady although maybe I just didn't stick around long enough.
What shady tactics are you referring to?
Developers easily fool themselves thinking they’ll save $9 p/m by building something from scratch in 3 weeks.
Oh no. The only thing that's high is our conviction that our BS meter is high. We fall prey to, come up with and promote as much BS as the next person.
And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
I think this is the best way to think of CoPilot. GitHub is selling it like its going to write all your code for you, but in reality it is just next-generation auto-complete.
That's not a bad thing. In some ways I'd argue its actually better. GitHub needs to change its marketing because even most developers seem to think that its out there to take away our jobs. Its not and can not. But it provides the smartest auto-complete you've ever seen and that can be useful, especially when wading through mundane parts of your codebase.
The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way.
The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways.
I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components!
Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy.
Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
It is true that more commonly it suggested at most a few lines of obvious code which could really only be written the way it suggested, but a number of people in the comments on this article mentioned using Copilot to come up with test cases, so I think people are actually using it to suggest larger snippets of code.
"GitHub Copilot blocks your ability to learn." Is a common refrain.
I don't see ANY industry-wide consensus on whether GitHub Copilot truly helps developers right now.
The only scenario I can get anyone to agree on is generating templates. Aka, JSON or CSS files that you then edit.
Others have already pointed out the case as a reply.
I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions.
The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
The sheer volume of subscription services I've signed up for as the CTO for a startup is mind-boggling. $8 here, $19 there, $49 for something important, $99 for something essential.
Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools.
Gitlab, Confluence, Jira, Asana, 1Password, co-pilot, codepen, sentry, jetbrains, gitlab plugins for jetbrains, Visual Studio, Docker Desktop, Perforce, Slack, etc;etc;etc;etc
Then there's things like Spacelift ($250!)!
The most frustrating thing is that:
1) I need to justify these expenses each for what value they bring, some things are nice to have but bring so little value on paper.
2) You can't just enable tools for some people, there's huge overlap and that overlap gets greater
I get that people need to be paid, but these things very quickly add up. I'm paying about 7-13% of peoples salaries already in these subscriptions, and I feel like a total dick for saying no to people or trying to consolidate these.
Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
Sounds like everyone has dodged a bullet!
A few tools that I put on the company card when I worked at a Big Tech Co as an IC:
* DataGrip (Jet brains)
* Colab Pro (Google)
* Postman Pro
These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was.
For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
* JetBrains (PyCharm professional, DataGrip, and Goland) ~$250/yr
* Lucidchart (Diagramming) ~100 /yr
* Paw (HTTP Client) ~$50 /yr
* Docker Pro ~$60 /yr
I think there's probably more, but I'm not at my work laptop to look, but those are the big ones. Those are only individual subscriptions. There's also huge costs when associated with things like Gitlab Premium ($20 - $100 /user/month), CI/CD, Code coverage tools, security scanners, etc. Companies pay A LOT for development tools.
If Kite thinks that the problem why no one will pay $9/mo for their service is because developers or their company's are cheap, they need to re-assess. The reason they couldn't sell their service is because it wasn't providing enough value to justify it. But companies are paying hundreds of dollars a month per developer in most cases for various tools. The extra $9 for Kite isn't the dealbreaker if there was enough value from it.
Maybe the product is poor. Maybe people didn’t believe the claim. But nobody said, “nah I don’t want my devs to be 18% faster.”
(I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.)
Developers will pay for software, if the value proposition is there.
I'd cut them some slack here. They had to get their team out of there first—with the way Putin is running things, they sure as hell couldn't announce they were leaving Russia until everyone who was going to follow them was out of there.
On the day of the invasion they tweeted a statement condemning the attack, and within two weeks announced they were leaving Russia.
https://twitter.com/jetbrains/status/1496786254494670851?lan...
https://blog.jetbrains.com/blog/2022/03/11/jetbrains-stateme...
Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
https://github.com/community/community/discussions/7553#disc...
https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_w...
That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter.
[1] https://www.ciodive.com/news/github-copilot-microsoft-softwa...
Copilot has a ton of still unresolved legal and compliance issues (copyright violation problems, sending proprietary code to Microsoft as you are writing it, etc.) and most larger businesses won't touch it with a 10 foot pole for that reason. There is even a class action lawsuit against Microsoft over Copilot already.
Indeed, I payed for a debugger because MSVC is pretty terrible
Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
I don’t like depending on something I could lose in a month or tethers me to the internet. I consider that more a service than a tool. I’d prefer to just buy something once that just works, but that business model might be dead too since people will pirate things that aren't tethered to some serverside component.
I guess what I’m saying is that I want to buy tools, but people are only renting. Personally I’m largely holding out hope this becomes someone’s open source passion project and I can truly own my tools.
Knowing examples such as Hudson CI & co, that probably makes it "no one", at a statistical scale.
The stuff most developers are comfortable paying for is things like hosting, tools that do something the developers find very boring or have no domain overlap and don’t have viable free alternative.
“Why would I pay 9.99 if I can set up a free alternative in a few days and host it myself for 4.99? If I can’t host it myself I don’t trust you anyway”
Mulling over business models, and noticing the 'devs won't pay' narrative in the blog post, it's interesting to see the existing business models in AI; basically they seem to be:
* API-driven cloud calls (this is a way to get high value out of your existing cluster if you're AWS, MS, etc.)
* Platform play + possible eventual lock-in: OpenAI/Microsoft
* Subscription service for very specific needs (Grammarly, writing support)
I wonder if engineers would pay $9.99/month (or even $49.99/month) for a 'grammar checker for PRs' - essentially: "Avoid embarrassing bugs before you commit". That is, I wonder if Kite could have been successfully sold as the third tier - sub service for something very specific.
I guess if it's a good idea, someone could pull the Kite repos and launch it -- but my guess is there may be a market in there.
Payscale estimates average engineering salary as having between $3,000 and $7,000 a month more in disposable income over writers -- and I would guess almost every professional writer pays for grammarly.
But, I agree that this is a new concept, and just spitballing -- right now, these sorts of linters and code formatting tools are mostly open source, so it would be some product marketing work to see if the market would actually pay.
"we were 10+ years too early to market, i.e. the tech is not ready yet."
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
Sounds like you know why people didn't pay for it. If it truly did make people as productive as you claim, it would have sold like hot cross buns on a cold day.
> making their developers 18% faster
If they're claiming 18%, it was probably more like 5% to 10% and it's really hard to sell something that's 5% better (especially when the alternative is free/ do nothing).
"just" the code you were working on, surely?
Any developer tooling company must understand this basic fact.
Having a tool that rapidly creates setters, getters, or even common algorithms in function/method bodies is neat, crucial even. But also a problem that has mostly been solved for decades now.
The actual difficulty, where software devs spend (or should spend?) most time is indeed in what you say "reading, reviewing, designing, arguing". Where I'd like to add that the "arguing/bitching" is crucial if done with the right people (stakeholders, business, etc: creating a domain -or ubiquitous- language).
No AI can help me with that. And the current AIs make that worse. Rather than learning and applying ubiquitous language, rather than evolving a clean, maintainable architecture, it blurps a generic(ish) blurp of code. That often has no place where it was suggested, is inconsistent, breaks encapsulation or coupling and so on. If you blindly accept all the suggestions, the code often becomes worse fast; but you do write a lot of lines of code quickly. Whoever cares about that, though?
Kite did sometimes offer some good suggestions in regular code, but it tried really hard to understand your code, and went belly-up when it didn't.
At that time, I tried some other ML-based autocompletion tool which wasn't specific to python, and which usually worked much better, except that it used far too much memory and caused regular crashes.
Maybe they improved kite since I tried it, or maybe "individuals don't pay for dev tools" isn't the whole story. Or maybe both.
Anyway, kudos for both trying and for open-sourcing the code at the end!
I do wish the Kite team all the best, and I hope they can re-use their skills in products that are commercially viable.
On the other hand, create an AI that can stand in during pointless meetings and the blank checks will shower down.
https://medium.com/swlh/kite-vs-tabnine-which-ai-code-autoco...
We have plenty of techniques that we know improve productivity (e.g. static types) but some people still don't believe it because it's really hard to prove productivity increases.
They were a VC backed start-up and it was indeed go big or go home for them.
As they seem to have identified, enterprise sales is one way to go big. Timing is also crucial. Right now, by iterating relentlessly over a decade on just the models, OpenAI and Cohere have commoditized access to the AI itself via APIs: Perhaps a better go-to-market than Kite's which had to build both the AI (ex: OpenAI Codex) and the end-product (ex: GitHub Copilot).
Even though they had 8 years to execute, they had zero leverage on the kind of network effects and developer mindshare that GitHub has or access to bottomless funding like OpenAI. Hindsight is 20:20.
This sounds like spite. Sure, copilot can be even better (what can't?) but it's already a great tool. It has a small learning curve (which is just getting comfortable with it) and then it can add a lot to your productivity. Of course, this is orthogonal to any copyright polemics out there.
Kite never got close to what copilot is.
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
So basically, everyone's fault but their own. Got it.
Edit: Also I want to say, that WE DO pay for stuff if it brings us value. Out of my pocket I pay for JetBrains, Github, Temius, and the SublimeText 4.
Throwing salt on the wound here but that’s just false. I mean, there’s copilot and it’s alternative that I can’t think of the name right now. more broadly there’s Jet brains ides, visual studio, Productivity apps, etc. look at product hunt or appsumo or popular show hns. Devs pay for tools, just not Kite.
Edit: I should clarify, enough devs pay for tools to make the market sustainable. Not all devs pay for tools.
As a CTO, most of my budget goes to paid services that add clear value with a clear value proposition. The value proposition with developer tools is usually quite murky. It's all very subjective and preference based. So, something like kite is a hard sell.
It's remarkable that they attracted so much investment. But of course that put them under enormous pressure to meet what were probably highly unrealistic revenue goals as well. That team might have made them a nice acquihire target at best.
(That manager getting the purchase order approved through corporate took months and months, but that's neither here nor there.)
https://www.ciodive.com/news/github-copilot-microsoft-softwa...
Yes.
It also has the nice effect of keeping the code available to the people most familiar with it, as they move on to other ventures.
It means that you didn't believe in open source while you were in business, and are only doing so now to score some points with your customers. There's no guarantee that someone will step up and maintain the project for you.
I went ahead and filed an issue on kiteco-public[0] about their derived data because the readme states:
> By the way, we are happy to share any of our open-source-derived data. Our Github crawl is about 20 TB, but for the most part the intermediate and final pipeline outputs are pretty reasonably-sized. Although please let me know soon if you want anything because we will likely end up archiving all of this.
However, I have no idea if this is the right way to contact them
They seemed like a really cool team, I wish them the best.
kite was that autocomplete solution that required you to have an account right? and they shipped your code to their servers? i remember trying it. some of us raised early concerns but our voice is not the loudest.
so again, the main problem is that kite was an intrusive solution for corporate networks. a developer needs to justify, through millions of layers, a solution like it. that it is safe to run it in a corporate environment.
why are you comparing yourselves to copilot? it's github!
not a single CISO will blink at trusting github, or microsoft, or google. a startup? it's not the kind of product that's helpful on a hobby project. the individual developer will pay where it makes sense. it makes sense in the corporate environment where there is tons of code to write.
so yeah, okay, that new terminal thing called warp. that autocomplete in the terminal called fig. you all ask people to create accounts and ship their data home? don't act surprised later.
I don't pay for Kite (or any other proprietary developer tooling like Github) because one day your company can choose to shut down, change its terms, or raise my prices and I'd be left without recourse, while also being locked in to a proprietary workflow. Just like you did today, which validates my hesistation.
Kite should have been open source from the very beginning. I hope the team can take away this learning for their next startup. I applaud teams like GitLab who build entirely in the open--and, as a result, have highly successful products and businesses.
I don't think the being locked in a proprietary workflow bit is your real reason, because when you break it down - this doesn't make much sense. Fear of needing to switch workflows down the line outweighs the [potentially temporary if company dies] boost in productivity?
Of course, this assumes kite fits your workflow well and you find it delivers value (you don't cancel immediately)
Meta's language models, GH Pilot, real life car auto-pilot. When it fails, it fails big. And the "we were 10+ years early to market" is just a big lie that bought them plenty of VC money. Good for them.
At least partially over. It's one of those things though; when you first see what's possible with neural networks it does get your hopes up. When you later realize the limitations, it's hard to walk back your old claims. Even Elon Musk has to realize that FSD is never going to happen by now. Google with all their learning and training data, still can't correctly find and smudge license plates or faces correctly on Google maps. If that much processing power cannot correctly identify two classes of objects, what chance do these cars have to classify tons more objects + adapt how they steer based on that information in real time?
It's kind of the explore-exploit dichotomy. You have some new technology (internet), in the first few years you have exploration and all the low hanging fruit are implemented, then everybody just starts iterating on similar ideas, which lead to less and less gain. The Uber/AirBnb/Amazon for X pitches. If you hear those you're in the late phase of the hype cycle. Because Y for X just means it's not a really new idea and plenty of people have thought of those.
Similarly you have some new technology like fuzzy logic, then some people thought of some good applications. But because the hype train was running it was put everywhere where it didn't make sense.
Or deep learning which was the first to have useful image processing. Now most research is tuning some parameters, adding compute, and hoping for better results.
But in the end we'll be left with some technological advances, and maybe in ten or twenty years somebody has a new idea which beats deep learning in learning efficiency.
- Sublime
- GitHub.com
- ACM Digital Library
(The latter two are subscriptions.)
Things I've payed for in the past that I no longer use:
- MS Visual C++
- Omicron Pascal
- Application Systems Modula-2
- Atari ST GFA BASIC 2.0
- Berkeley YACC and FLEX port to TOS/GEM
- ...
Overall, many dev tools are free nowadays, which creates an expectation, perhaps, that it should all be free (I disagree in principle, but of course it is nice to see this trend progressing).
I appreciate that Kite is posting a post mortem for others to learn, and I wish they had been able to find a niche where people pay for their work. I love software tools as a work product, but have been told by many experienced people it's not a good area for making money.
The problem with Kite seems that their engineered first ("This machine learning AI is so cool, what can we do with it?" "I am a VC, are you doing AI? TAKE MY $$$$!") and only after burning through millions started to look at how to actually make money out of it.
And discovered that:
a) Hobbyist/individual developers rarely want to pay yet another subscription (can't justify it if you aren't making money with it & even $10/month subscriptions do add up!)
b) Corporate developers don't have purchasing authority. Everything must get approved, by both accountants and legal/compliance. Expecting a large company to pay a huge monthly/annual subscription fee for what is essentially a better autocompleter? Good luck with that.
That "Oh but your developers will be 18% faster!" argument is BS. 90% of the corporate developer's time isn't spent on typing code but on debugging, design, maintenance and meetings. Kite (or Copilot) don't help with that.
c) What about copyright/compliance issues? This has been trained on Github repositories - i.e. the same as Github's Copilot. How do I know where does the completed code come from? What about licenses on that code? Can I filter only for permissive/non-contagious (i.e. non-GPL) licenses? What about my code/whatever I type? Does it get sent to your servers? That alone is a complete no-go for companies.
In other words, a classic case where one shouldn't ask whether something could be done but whether it should. Someone outside of their engineering bubble and with a bit of business acumen would have told them that. Or at least told them to do a market research first, before spending all that time and money.
But hey, they had a good ride for the VC's money and are winding it down in an organized manner, without leaving a ton of shattered lives and a mountain of debt behind. So that's a plus.
This is the one-sentence summary about why the business failed, but it's kind of a strange way of putting it.
I am dead sure that there were plenty of advisers along the way who told the company's executives that its monetization plan was weak and unlikely to succeed. But everyone assumes that they'll be the exceptional case.
"It took too long to figure that out" makes it seem like the most likely scenario wasn't staring them in the face the whole time.
From what I remember, people got super annoyed at Kite for placing ads in open source projects and they just never caught on.
I see CoPilot all around me and it's generally well regarded and pretty widely adopted given how new it is.
Is there any data for this statement you can share?
(Thanks for working on kite and good luck!)
Credit to where credit is due. I worked in R&D here in a group tackling it for almost a decade ("program synthesis"), and while Copilot has a lot more to do, it solved so much of the usability & basic use case gap of what the R&D community had been attempting for years. Large language models & transformer models have been out for years, and the Github team executed well on adapting them.
(Separately: There _is_ an interesting question whether this space is VC-investable -- how likely will at least 1 startup here make it to 9-10 figures of revenue. But that's another story.)
It's not new, but the requirement to pay for it is.
Even plain old Jedi was a decent competitor to Kite.
So you we're no beat by billion-dollar CoPilot I'm afraid...
True, AI assisted coding does not deliver 10x. But as a user of another AI assistant, I feel that it gives me ~1.25x to ~2x improvement for the keyboard typing when I code. And that is respectable too :) AI for me currently allows me to tab complete some things that previously an IDE on its own was not able to.
Tabnine for JetBrains CLion. It works with Rust and several other languages.
Depending on how local he's talking, this isn't really true of Copilot. In my experience it will use context all the way up to the top of the file, even in very long files. And at least the Rust version even seems to look at the imports—if you have a use declaration it will actually correctly build and use structs in other files regardless of whether you've yet used them in the current file.
I can imagine this is easier in a language like Rust that has a really strict module system, and to be fair the project that I've been using it on is a side project that isn't over 10,000 lines of code yet. If I were up to 30 imports per file I can imagine concatenating would become much less effective.
My understanding is that copilot is largely a self-supervised approach. They feed a massive body of (somewhat noisy) code into the model. The model really does learn a lot of structure on its own and this is a testament to deep learning on noisy datasets.
I'm guessing the "hooks" that they have already from IDE's, language-servers, etc. _are_ quite "structure-aware" - so they want the predicted structure as well as the code, so they can improve the typing experience beyond line-completion.
I think the estimation of 100 million for such a task is maybe too high? I don't know - it feels like you could actually get quite close to such a system by simply using thousands of custom prompt engineering tricks that prepend structure examples to the prompt?
From an old programmer perspective, it doesn’t make much sense, but a new programmer will not want to learn the old way, which will be effectively obsolete from lack of updates. If there’s any value to be derived from it, perhaps it is demand for hardware that will run enormously-inefficient code. The way that now you see people doing full sorts to get the third-largest value just because it’s easier to write it that way, you will see code that also does analytics and builds a distributed hash table to accomplish the same task, just because more capability means more usage means more suggestions to carry along that code.
I think it was a mistake to think of computer programs as a linear text language, but I don’t see this turning back. At some point, the concept of programming a machine will merge entirely with the method of interacting with a machine, which is to say, communicating intent, and then I suppose we can relax into a very comfortable full-service 5-star extinction.
You need to have 500 users to understand that, not 500K. A well-written postmortem otherwise.
When I read “non-local context,” it really drove home for me just how off the mark they were and changed the whole tone.
It also makes me think were they just hoping the solution would fall out of the sky? Seems irresponsible if that was part of their calculus.
This seems to be very much the standard story for "AI"; not quite there yet. Given the history, it's, er, surprising that people are constantly surprised by this.
I wish more projects would do this
It's also demoralising to see an entire category form without you, especially when you were working tirelessly towards it early on. I've really learned this the hard way also. Good luck to the Kite team in their future endeavours.
Are there a lot of businesses where individual developer productivity, with a narrow definition of LOC per hour, is the bottleneck?
I've worked for 10 years as a web dev and the bottleneck is very often at the product management level (tickets not ready, goals changing, haven't got the credentials for the 3rd party API yet..) and a minority of the time it's my brain (yes sometimes I need to think before I write code). It's rarely how fast I can write a function. So if you make me 18% faster at something I do 1% of the time... good luck making money out of me
Anecdotally, this was my experience at many companies before working at FAANG. But it's not my experience now.
They got wiped out by microsoft, github copilot, and litigation issues around AI provided code.
I was never aware they changed the architecture to keep code analysis entirely local. I would have purchased a subscription, had I known.
I agree that Kite didn’t deliver the 10x. I was an early user and tried hard to use it but didn’t find the benefit compelling enough to drop into my workflows, but it was very exciting.
I’m sure I speak for all of HackerNews when I wish you the best for whatever is next for the team.
Also, what are you good folks doing next?
I’m an individual developer and I pay for tools all the time. They just have to be of value to me. If developers weren’t paying for your tool, maybe look within.
I can't think of any dev tools I've paid for in this century; it seems better to rely on open source infrastructure.
Maybe they should have spent more budgets on marketing.
That said, I agree that no one wants to pay for developer productivity. The only exceptions are IDE and databases.
Anyone who's either worked at a developer tooling company or tried to sell to developers themselves (I personally did both, having worked at CircleCI in the past and now building my own developer tooling product at https://reflame.app, Show HN launch thread here: https://news.ycombinator.com/item?id=33134059) can probably back the observation that we individual developers are notoriously reluctant to open our wallets, even for products that we love and use daily, despite our high disposable income relative to professionals in other markets.
Gonna share a few of my own hypotheses for some of the contributing factors as comments below for discussion.
Would be fun to see folks share their own! Especially if you've seen successful strategies for how someone might be able to overcome these hurdles to selling products to individual developers at scale (a topic near and dear to my heart these days)!
Again, I'm totally guilty of this myself, since Reflame started as a side project initially to scratch my own itch, and I can't claim to have done an exhaustive search on the problem space before I started.
This results in a vicious cycle where every product, however innovative it might be at its inception, gets quickly commoditized by dozens of similar products immediately following any signs of traction, so they end up having to shift to competing on price eventually.
Combined with https://news.ycombinator.com/item?id=33691132 means any product that isn't available for free then eventually rots into obscurity due to the unfair distribution advantage of "free" in this market. Thus they are forced to offer a free version themselves and the cycle continues.
I certainly suffered from this myself to a rather extreme degree in the past, having categorically refused to pay a single cent for anything I used to build side projects with, until I started to seriously think about pricing for my own product. Eventually I realized throwing money at almost any problem where it could buy me more free time should be a no-brainer considering how highly I value my free time.
Tangentially, I think there's an interesting analogue in here to what Steam did in the PC games market, but I digress...
Most SaaS are priced completely independently of infra costs (or any other costs really), but we are much less likely to accept products priced with high margins on top of obviously low infra costs, even though that doesn't represent nearly the full cost of running a SaaS (which consists mostly of payroll due to how high our salaries typically are haha...).
We also like to justify this line of thinking by the argument that "well, I can build this myself in an afternoon" (significantly underestimating the real ongoing time investment required to build and maintain a SaaS product, even seemingly trivial ones) or "I can write some bash scripts and put this on the VPS that I'm already running anyways" (undervaluing our own time).
Most people won't pay for a ton of small services since it adds up. There is a minimal threshold to pass to make online transactions financially reasonable, making the pricing models make little sense for most services. Given that most cheap services (ie. those at or below $5/mo) don't have large infrastructure costs, it's an even harder sell. Not to mention that sometimes people rather watch movies or television than get tools to make them more productive.
This is doubly compounded when you look at opportunity costs. With the amount of software that can be self hosted, the costs isn't just the comparison of having the tool or not or even other developer focused offerings, but instead having this tool (or access to it) versus any other tool that can be self-hosted (including those that might not exist yet).
Also, lets say we spend $5/mo, well for that we can host our own server which can easily be used for more than one purpose (with WireGuard now even the smallest VPS can be used to saturate most home links and bypass CGNAT easily). Increasing the monthly spending just increases the amount of opportunities.
This of course doesn't touch the elephant in the room which is privacy.
For the regular user privacy isn't a huge deal for these small services. For the average reasonable user that doesn't upload sensitive data, at worst it would be something like a personal photo being seen.
On the other hand as a software developers using these tools is a lot more complicated. Licensing, copyright, and possible work contracts start to matter. If the service interfaces with code (like Kite or GitHub Co-pilot) then you get into a some serious murkiness due to the fact that you don't really know what they are doing with it on their end. Even the things like telemetry and what type of data is being sent back matter in corporate environments.
I've recently noticed a couple of companies open sourcing their product upon discontinuing the company -- is this a new trend or has it happened for a long time and I am only just recently noticing it?
> Then, our product failed to generate revenue. Our 500k developers would not pay to use it.
Isn’t it better to work with a smaller number of users but more closely first. Otherwise you burn the chance to impress all of those users. Plus with 100 users you have a decent sample but you can also reasonably interview them all one on one.
Now whether or not that's due to the fact that copilot had the financial resources to train a significantly superior ML model is another question, but throwing shade at copilot is a fairly transparent move.
I highly doubt that you failed! You blazed a trail forward for people in the future to follow. Financial success is not the same thing as taking a super tough problem to solve and then making inroads solving or starting to solve the many sub-problems (and their sub-problems) that invariably show up as a result of taking that path.
>"Then we grew our user base. We executed very well here, and grew our user base to 500,000 monthly-active developers, with almost zero marketing spend."
That's extremely impressive in my book! (By comparison, I failed to get 2 users -- for one of the apps I built -- and that was with marketing spend! <g>)
>"Then, our product failed to generate revenue. Our 500k developers would not pay to use it."
You might mean that there may have been an issue with communicating the VALUE of your product such that users would "see" (magical word, "see" -- "percieve", "understand", "observe in a way that you do") the VALUE of it -- such that they would be willing to equally-and-oppositely exchange their money for that VALUE...
Finally:
I do not think that you failed, and you have no reason to apologize to your investors, customers, employees and others.
You pushed the envelope -- and you created great value for future generations who will no doubt benefit from your pioneering steps in this gargantuan undertaking.
Well done -- and I think more people should appreciate you for that!
That being said, glad to see a lack of Our Incredible Journey type language here and more of a true postmortem of their business and technical decisions. It is rare to see a company go into so much detail when shutting down.
* Our diagnosis is that individual developers do not pay for OUR tool.
What sounds more plausible you:
- engineering managers hate free money
- it’s obvious to everyone that this statistic is bullshit
Sure, your people were great but they didn't innovate enough to make an attractive product (granted, AI code autocompletion is hard - I doubt we'll get something I'd be happy to pay before we reach GAI and we'll be all out of a job by then).
Oh and the "It's not the tech fault which is amazing, it's just a sales pipeline issue!"
Look, I understand caring about your employees and I said the same BS when my company failed trying to shift all the blame on me and not on my team. When you are in a startup it's everyone's job to say "hey, btw, what we want to do will suck because the tech is not there".
If you see something raise it and try to pivot, or you'll be out of a job with worthless grades.ß in
Maybe you could have cut your losses earlier on.
Many people I know find copilot extremely helpful. I think tools like it are about to become extremely important to the productivity of everyday developers. I seriously doubt it will take $100M to develop. The company Kite might have needed $100M to get there, but I bet you a few smart people working evenings in their garages can get there too.
Also the "nobody pays for dev tools" line is pretty obviously a weak excuse. Github is a developer tool that was worth $7B+. The truth was they just didn't provide _enough_ value to get people to pay for it. That's clearly true, and goes along with their idea that they were too early. Not that the problem is impossible.
I am very impressed and happy to see the open-sourcing of their code like this. I often find myself thinking about how much human knowledge and effort disappears when a company shutters and all of their documents, code, etc go with them.
Sadly, I never heard of Kite until copilot came out. As someone who pays for tools, I would have considered it (have paid for JB for years, various atlassian tools, and other utilities/etc).
Whoever wrote this will go far.
Quite simply developers are not decision makers. Often engineering managers aren't even decision makers. I fully believe if you want to dominate in that area you need to target decision makers who force it upon their developers. How many of us have been told we're using X database or we're using X project management tool or even X virtualisation system? Management makes these decisions which is why if you go an AWS conference you'll find majority of the people there aren't techies but management and lots of the talks are aimed at management understanding the tech.
I've seen many other commenters lament the fact that "even though devs make a boatload of money, they don't want to pay for their tools".
This may be true.
But I think the biggest issue is that most developers are only developers "at work". And I've seen far too many people work with subpar tools (old, dingy PCs that would take ages to do anything). Management thinks this is fine, and that a 5 yo intel U laptop with 8 GB RAM is AOK for running heavy computations in 2022.
So, going out of their way to buy some "text editor" when vscode is free? Not gonna happen. Even if the devs themselves are convinced.
Not to sound overly mean but it might have been a good idea to start with testing this idea first/earlier. Additionally, it seems to me like they didn't do a great job at identifying their customer. It's probably not individual devs but rather the people they work for. So you're in a B2B business and need to sell it that way.
The meaner response would be that it seems like developers do not pay for YOUR tools. Seems like there are plenty of paying customers for copilot for example.
What?
You're supposed to create the technology, not wait for others to create it. That's why VCs give you money, isn't it?
> We built the most-advanced AI for helping developers at the time, but it fell short of the 10× improvement required to break through because the state of the art for ML on code is not good enough.
Aren't you supposed to advance the state of the art?
> but the problem is very engineering intensive
So you weren't a technology company?
Gary Marcus has been doing a good job of exposing this: https://garymarcus.substack.com/
Another key piece of evidence, the failure of all FSD attempts trying to use ml/dl thinking its more than just statistics.
Disagree. I pay for Visual Assist as an individual because its a huge productivity booster. I suspect the issue is pricing vs perceived value.
> It includes our data-driven Python type inference engine
I couldn't find which repository this lived in. I am very interested in it, as my team maintains a few open source static analysis and code generation tools. We'd be interested in trying this out.
Copilot ended up being rarely helpful for me. But on the other hand, MS IntelliCode (I think it only works for C# in full Visual Studio) was a fantastic productivity tool that actually sped up writing code because it does understand the structure of C# and your codebase. Wish it was available for other languages and VSC
Counterpoint:
Maybe instead of blaming potential customers for not finding enough value for your product, you might need to start looking inward.
I gladly paid for my own personal license for R# that I kept across four jobs over 8 years. I only stopped paying for it because I no longer develop in C#.
My favourite use is at the command line. It's great!
I pay for it myself and use it in all sorts of contexts.
EDIT: Actually, perhaps that is actually smart. If you want to find people who would pay for dev software you probably should target people who pay for dev software already. Jetbrains is better for paid plugins than VS Code by this logic.
> We failed to build a business because our product did not monetize, and it took too long to figure that out.
Yet people always defend telemetry in software, saying it's how they improve their product. 7 years of telemetry, and they couldn't figure it out?!
A lot of work I think and as is pointed out here devs wants their tooling for free.
- developers willing to use AI crutches instead of their own brains to write their code;
- developers unwilling to pay other developers while being paid themselves by companies whose profit models are often far removed from the honest craft of developing something wholesome.
Not sure about Kite though
Hopefully the next project goes well!
They do pay for tools, but not enough to make it a full time. I've got a few "side" projects that bring in a few K/month each.
I read about and tried evaluating Kite at the time and it seemed like it was in some kind of private invite stage. I remember thinking it must have been acquired and wasn’t taking new users. This must have been an incorrect take.
I’m surprised Tabnine is not mentioned in this thread at all, though because that was acquired and afaik is still operating.
Before copilot came along, Tabnine, not Kite, seemed like the ai took to beat.
I also remember a Python dev relations person from Jetbrains going on a podcast and clowning on AI code completion. That was in April of 2021. [2] A month later copilot dropped.
The very strange thing about that was Jetbrains described efforts to build an ML-based code completion plug-in in 2016! [3] It obviously failed to follow through on that.
I still think G Co pilot represents a threat to jetbrains IDEs overall. Even the packaged autocomplete can’t compete on basic stuff copilot does now.
I disagree with the idea that AI code completion is not good enough yet. I see that said all the time and yet it can masterfully fill in boiler plate today.
It can be way better, particularly in languages outside JavaScript and Python, but it’s usable now and maybe even profitable as a service if the business is not leveraged by VC capital.
If you listen to the September interview with Eddie Aftandilian of Github Copilot you would realize how early it still is for that product, as how to measure success in code completion is something still requiring behavioral patterns that are still being recorded.
Here’s the episode, listen 20 mins in: https://www.se-radio.net/2022/10/episode-533-eddie-aftandili...
[1] https://news.ycombinator.com/item?id=25074393
[2] https://twitter.com/jetsetter/status/1379438096232587265
[3] https://blog.jetbrains.com/idea/2016/09/share-your-stats-to-...
copilot: "Get code suggestions in more than a dozen coding languages including Python, JavaScript, TypeScript, Go, and Ruby", how about c, c++ even lua here? if they cover c and c++ I can pay $10 per month right away.
Builders buy hammers, drills. We should be parting with our money for tools that multiply our earning capacity as contractors and consultants.
I would pay 5x as much for many of the tools I buy too, such is their value.
Is it easier to build AI for pure functional programming languages?
Does it mean employees don't pay for the tools? Or that single-person ("individual") independent developers don't?
Why wouldn't a developer pay $100 for a tool that saves a day of work for them?
Because their boss will give them something else to do and now the developer has 100$ less and nothing to show for it.
The common definitions have to do with stealing, but an equally valid definition of the word hijack is to:
> take over (something) and use it for a different purpose.
Taking over a project so you can have it to advertise your service is exactly that.
> many programmers would consider [this] a violation of the open-source spirit.
Instead of making the untrue statement above, just say
"They used, in my opinion, an unethical way to advertise their product; specifically, they bought OSS products and put their ads in there."
And you know that the company is supposed to buy this stuff and you can get fired for using unapproved tools that send code and probably violate copyright.
They are clearly useful but people still don't want to pay.
Good riddance to bad rubbish
First, we failed to deliver our vision of AI-assisted programming because we were 10+ years too early to market, i.e. the tech is not ready yet.
We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough. You can see this in Github Copilot, which is built by Github in collaboration with Open AI. As of late 2022, Copilot has a number of issues preventing it from being widely adopted.
The largest issue is that state-of-the-art models don’t understand the structure of code, such as non-local context. We made some progress towards better models for code, but the problem is very engineering intensive. It may cost over $100 million to build a production-quality tool capable of synthesizing code reliably, and nobody has tried that quite yet.
Nonetheless, we could have built a successful business without 10×’ing developer productivity using AI, and we did not do that.
We failed to build a business because our product did not monetize, and it took too long to figure that out."
That's not the same thing as being too early to the market. That simply means you didn't have a solution capable of solving a problem.
lol
$100 million is nothing tbh.
How many working hours, tons of materials will it take?
Could they be used on more relevant needs to humanity?
If so, why use them on a moonshot we will die before humanity achieves? Essentially approving burning up resources on unverifiable, perhaps unachievable outcomes?
We still fund a lot of agency to iterate on high minded potential as if future humans have an immutable obligation to carry on the work.
Secular norms and justifications are not sacrosanct. The dollar is a made up token of power.
Tbh even at tech giants that make tens of billions of dollars, a $100 million investment is likely a lot, I'm guessing this sort of investment will require sign-off by CEO or at least VP level along with a solid business plan.