I know this first hand, building a developer tool startup and failing to reach any level of revenue. In the end, the tech was bought out by a larger company to recover a fraction of our VC investment.
The challenge is that when you're building software for developers, they already know how it must work.
It's like trying to sell magic tricks to magicians. Sell magic to regular people, and you'll see some significant revenue.
I've used Kite before. It was ok. But I am a SWE. It's entirely possible that Kite would have seen major adoption if the push was towards non-technical folks trying to get their feet wet in software. Eg: Data scientists or business.
The reason why BI tools sell so well at the moment is that you have tons of C-level execs that like the appeal of a business-optimizing tool requiring little to none of any actual software development.
Let that be a lesson to everyone. You can't blow away developers. They're just too damn ~~smart~~ well-informed.
Edit: Another anecdote: A buddy of mine built a bespoke OCR and document indexing/search tool. He has ~60 paying clients (almost exclusively law-firms and banks) that primarily work with printed pages on paper. No Saas. No free tier. The client data resides on an on-premise Windows box, avoiding issues with sensitive data in the cloud etc.
He's a solo dev with support contracts and nets something like $1000/month from each client.
For your average lawyer/paralegal, the ability to locate and reference a single page from thousands of pages in under a second is magic. So they pay for it wholeheartedly.
Sublime Text.
I sat through scores of interviews and pairing sessions with developers back when Sublime was a thing and the vast majority (>90%) of devs would rather exit out of that pop-up asking for support then pay the measly $30 or whatever regardless of their massive incomes and increased productivity that Sublime brought them.
We developers are no more altruistic than anyone else regardless of the lies we fed ourselves in the early days of FOSS, internet, bitcoin, etc.
:(
- time
- headache
- improve my quality or quantitive results
Software developers very simply would rather build their own half assed solution to a problem rather than pay for a half assed solution.
Offer quality, we'll pay.
The other thing is they have to be tools I want to use. I am an outlier I am sure. I hear often "let your employer pay for it" but they don't always necessarily pay for the tools I need to use. Having my own JetBrains license grants me strong freedom.
Jetbrains' pricing model is also good, they reduce price each year (until 3rd), so you get rewarded for having a long term subscription. If you break commitment you get the regular pricing and you start over.
I remember trying Kite, but I removed it once I saw the pricing. It was more expensive than Jetbrains IDEs (which are less than 2$ a month for individuals when you pay in a bundle - 149$ at the time) which bring much more value for the money. For me it didn't make sense to pay 20$ for just incremental improvement (if even that) over Jetbrains Intelisense.
Hijacking the quote...
I can't count the number of times I see well-paid developers using the Sublime Text trial.
A bit tangential to the original post, but where does this belief that data scientists are non-technical folks? I am a data scientist myself, and in my view it's way more technical than most software development. Albeit I wouldn't still call neither data scientists nor software engineers "smarter" than the average.
Sure, if you want to train your bread and butter text classifier it just takes 10 lines of boilerplate code. But you don't need an AI-assisted tool for that - you just go to hugging face, copy paste those 10 lines, done, it's certainly faster than getting some AI-assisted code editor work for you.
For everything that is a bit more complicated, you need endless adjustments to your code, and it's quite unlikely more than a handful of people before you ever wrote the same code. It is, indeed, a somewhat painful and slow process (because just "testing" your code often takes minutes, if not hours, so finding out bugs becomes annoying). And a somewhat simple, AI-based, error highlight tool might be useful to weed out the most stupid ones and save some time.
But I will never trust something like copilot (or Kite, I guess, which I never tried) to write my code for me, as the challenging parts of the work involve long-term connection between different pieces of code (data loader, loss function, model function) that are written independently but must "cooperate" in a very non-trivial way. It is not at all uncommon that I make hours-long screen sharing calls with a colleague, discussing non-trivial mathematical computations, only to end up changing one or two lines of code that don't have an immediate link with the problem we are trying to solve.
This kind of things are notoriously hard for AI to grasp, so they can't do any decent job in writing that for me. Add on top that a lot of the code you find freely online is just ridiculously bad or broken, and you might only get unusable models generated by AI engines trained on those.
So, what kind of work are you referring to when talking about "data scientists or business" in your comment?
> I know this first hand, building a developer tool startup and failing to reach any level of revenue.
Just because your startup failed doesn't mean an entire category is unsustainable.
I've been living from sales of a developer tool for the last 10 years, and there are plenty of other paid developer tools out there that show developers absolutely do pay for developer tools.
Now, maybe some of the startups have unrealistic expectations. A Python documentation reader probably wont turn into a billion dollar revenue company no matter how smart it is.
But I'm pretty sure there is a market for dev tools. Maybe the market is smaller, or harder to crack than you thought, but saying "there is no way" isn't going to help anyone.
I believe this is incorrect. I pay for many tools, but I would not pay for Kite. The problem is not that developers don't pay for tools, but that Kite, or AI-assisted code, does not address a pain point. It's a slight improvement, but I don't feel pain when I need to write code without it.
That's different from something like CI tools that I pay for. When I need to wait long for CI to finish I get annoyed. That's when I pay.
I tried Kite a few years ago but didn't feel like I was getting much value out of it and never payed for it.
In contrast, I started paying for Github Copilot as soon as it was no longer free.
¯\_(ツ)_/¯
Just FYI, magic tricks are basically only sold to magicians. There's a thriving market of magic shops, especially online, where magicians go to buy new tricks (in the form of books/videos), new "gimmicks", etc.
I'd wager that a significant portion of all magic that is done is actually by magicians, for magicians, and partly in order to sell magic tricks.
I think it's just domain awareness, and the "they're smart" trope needs to be dismantled.
I think it plays into the technocracy problems we have now. We can solve it, we need more tech. More more more. People think we can solve social/political problems with tech - insidious.
I'm sorry it hasn't worked out for them, but they get my respect for this unusually frank self-assessment, real humility, and following through on the fine words with the actions of sharing their tools they built. They achieved a lot and I hope their future endeavors are wildly successful.
Do you mean that what you built didn't worked as it should? I don't understand, I've paid multiple times for tools that I find useful, even if they weren't perfect.
This misconception has been promoted by companies with an interest in promoting their platforms, using the expeditive procedure of subsidizing (often inferior) tools, with the collateral effect of making impossible for tools vendors to compete.
But by no means it's a law of physics. Make something programmers want. It's weird how little have the tools improved in twenty years.
That was a really shocking insight for me. We do not own our means of production. And suffer all the textbook consequences that follow.
Maybe unions could help with that. Imagine using union-funded licenses, compute, storage etc. to experiment with your side projects, build your prototypes without risk of losing IP to your current employer.
However, I would never pay for stuff that I can get free. I could talk to my company to buy it, but I would settle for something close and free if it come to that.
The blog post was quite clear on the ”the tech doesn’t work” part, which seems like a more likely reason for their demise. Selling developer tools is hard, but selling non-functional tools is exponentially harder
I never used Kite, but I've tried Github Copilot twice, and found it marginal at best (and distracting at worst - which is why I turned it off both times). If Kite was similar, the reason I'm not paying is that coder AIs are not providing any value.
Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.
Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.
The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.
My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
Since we literally build all of this our B.S. detection meter is really high.
Kite thought it can go after the up and coming new developers by doing slightly shady things.
However, developers also have an incredible allergy to such tactics and it forever taints your brand.
So overall, developers do pay for tools, just not useless ones with shady growth tactics.
And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way.
The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways.
I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components!
Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy.
Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions.
The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
A few tools that I put on the company card when I worked at a Big Tech Co as an IC:
* DataGrip (Jet brains)
* Colab Pro (Google)
* Postman Pro
These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was.
For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
Maybe the product is poor. Maybe people didn’t believe the claim. But nobody said, “nah I don’t want my devs to be 18% faster.”
(I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.)
Developers will pay for software, if the value proposition is there.
Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
https://github.com/community/community/discussions/7553#disc...
https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_w...
That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter.
[1] https://www.ciodive.com/news/github-copilot-microsoft-softwa...
Indeed, I payed for a debugger because MSVC is pretty terrible
Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
I don’t like depending on something I could lose in a month or tethers me to the internet. I consider that more a service than a tool. I’d prefer to just buy something once that just works, but that business model might be dead too since people will pirate things that aren't tethered to some serverside component.
I guess what I’m saying is that I want to buy tools, but people are only renting. Personally I’m largely holding out hope this becomes someone’s open source passion project and I can truly own my tools.
The stuff most developers are comfortable paying for is things like hosting, tools that do something the developers find very boring or have no domain overlap and don’t have viable free alternative.
“Why would I pay 9.99 if I can set up a free alternative in a few days and host it myself for 4.99? If I can’t host it myself I don’t trust you anyway”
Mulling over business models, and noticing the 'devs won't pay' narrative in the blog post, it's interesting to see the existing business models in AI; basically they seem to be:
* API-driven cloud calls (this is a way to get high value out of your existing cluster if you're AWS, MS, etc.)
* Platform play + possible eventual lock-in: OpenAI/Microsoft
* Subscription service for very specific needs (Grammarly, writing support)
I wonder if engineers would pay $9.99/month (or even $49.99/month) for a 'grammar checker for PRs' - essentially: "Avoid embarrassing bugs before you commit". That is, I wonder if Kite could have been successfully sold as the third tier - sub service for something very specific.
I guess if it's a good idea, someone could pull the Kite repos and launch it -- but my guess is there may be a market in there.
"we were 10+ years too early to market, i.e. the tech is not ready yet."
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
Sounds like you know why people didn't pay for it. If it truly did make people as productive as you claim, it would have sold like hot cross buns on a cold day.
> making their developers 18% faster
If they're claiming 18%, it was probably more like 5% to 10% and it's really hard to sell something that's 5% better (especially when the alternative is free/ do nothing).
"just" the code you were working on, surely?
Any developer tooling company must understand this basic fact.
Having a tool that rapidly creates setters, getters, or even common algorithms in function/method bodies is neat, crucial even. But also a problem that has mostly been solved for decades now.
The actual difficulty, where software devs spend (or should spend?) most time is indeed in what you say "reading, reviewing, designing, arguing". Where I'd like to add that the "arguing/bitching" is crucial if done with the right people (stakeholders, business, etc: creating a domain -or ubiquitous- language).
No AI can help me with that. And the current AIs make that worse. Rather than learning and applying ubiquitous language, rather than evolving a clean, maintainable architecture, it blurps a generic(ish) blurp of code. That often has no place where it was suggested, is inconsistent, breaks encapsulation or coupling and so on. If you blindly accept all the suggestions, the code often becomes worse fast; but you do write a lot of lines of code quickly. Whoever cares about that, though?
Kite did sometimes offer some good suggestions in regular code, but it tried really hard to understand your code, and went belly-up when it didn't.
At that time, I tried some other ML-based autocompletion tool which wasn't specific to python, and which usually worked much better, except that it used far too much memory and caused regular crashes.
Maybe they improved kite since I tried it, or maybe "individuals don't pay for dev tools" isn't the whole story. Or maybe both.
Anyway, kudos for both trying and for open-sourcing the code at the end!
I do wish the Kite team all the best, and I hope they can re-use their skills in products that are commercially viable.
On the other hand, create an AI that can stand in during pointless meetings and the blank checks will shower down.
They were a VC backed start-up and it was indeed go big or go home for them.
As they seem to have identified, enterprise sales is one way to go big. Timing is also crucial. Right now, by iterating relentlessly over a decade on just the models, OpenAI and Cohere have commoditized access to the AI itself via APIs: Perhaps a better go-to-market than Kite's which had to build both the AI (ex: OpenAI Codex) and the end-product (ex: GitHub Copilot).
Even though they had 8 years to execute, they had zero leverage on the kind of network effects and developer mindshare that GitHub has or access to bottomless funding like OpenAI. Hindsight is 20:20.
This sounds like spite. Sure, copilot can be even better (what can't?) but it's already a great tool. It has a small learning curve (which is just getting comfortable with it) and then it can add a lot to your productivity. Of course, this is orthogonal to any copyright polemics out there.
Kite never got close to what copilot is.
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
So basically, everyone's fault but their own. Got it.
Edit: Also I want to say, that WE DO pay for stuff if it brings us value. Out of my pocket I pay for JetBrains, Github, Temius, and the SublimeText 4.
Throwing salt on the wound here but that’s just false. I mean, there’s copilot and it’s alternative that I can’t think of the name right now. more broadly there’s Jet brains ides, visual studio, Productivity apps, etc. look at product hunt or appsumo or popular show hns. Devs pay for tools, just not Kite.
Edit: I should clarify, enough devs pay for tools to make the market sustainable. Not all devs pay for tools.
As a CTO, most of my budget goes to paid services that add clear value with a clear value proposition. The value proposition with developer tools is usually quite murky. It's all very subjective and preference based. So, something like kite is a hard sell.
It's remarkable that they attracted so much investment. But of course that put them under enormous pressure to meet what were probably highly unrealistic revenue goals as well. That team might have made them a nice acquihire target at best.
It also has the nice effect of keeping the code available to the people most familiar with it, as they move on to other ventures.
It means that you didn't believe in open source while you were in business, and are only doing so now to score some points with your customers. There's no guarantee that someone will step up and maintain the project for you.
I went ahead and filed an issue on kiteco-public[0] about their derived data because the readme states:
> By the way, we are happy to share any of our open-source-derived data. Our Github crawl is about 20 TB, but for the most part the intermediate and final pipeline outputs are pretty reasonably-sized. Although please let me know soon if you want anything because we will likely end up archiving all of this.
However, I have no idea if this is the right way to contact them
They seemed like a really cool team, I wish them the best.
kite was that autocomplete solution that required you to have an account right? and they shipped your code to their servers? i remember trying it. some of us raised early concerns but our voice is not the loudest.
so again, the main problem is that kite was an intrusive solution for corporate networks. a developer needs to justify, through millions of layers, a solution like it. that it is safe to run it in a corporate environment.
why are you comparing yourselves to copilot? it's github!
not a single CISO will blink at trusting github, or microsoft, or google. a startup? it's not the kind of product that's helpful on a hobby project. the individual developer will pay where it makes sense. it makes sense in the corporate environment where there is tons of code to write.
so yeah, okay, that new terminal thing called warp. that autocomplete in the terminal called fig. you all ask people to create accounts and ship their data home? don't act surprised later.
I don't pay for Kite (or any other proprietary developer tooling like Github) because one day your company can choose to shut down, change its terms, or raise my prices and I'd be left without recourse, while also being locked in to a proprietary workflow. Just like you did today, which validates my hesistation.
Kite should have been open source from the very beginning. I hope the team can take away this learning for their next startup. I applaud teams like GitLab who build entirely in the open--and, as a result, have highly successful products and businesses.
I don't think the being locked in a proprietary workflow bit is your real reason, because when you break it down - this doesn't make much sense. Fear of needing to switch workflows down the line outweighs the [potentially temporary if company dies] boost in productivity?
Of course, this assumes kite fits your workflow well and you find it delivers value (you don't cancel immediately)
Meta's language models, GH Pilot, real life car auto-pilot. When it fails, it fails big. And the "we were 10+ years early to market" is just a big lie that bought them plenty of VC money. Good for them.
At least partially over. It's one of those things though; when you first see what's possible with neural networks it does get your hopes up. When you later realize the limitations, it's hard to walk back your old claims. Even Elon Musk has to realize that FSD is never going to happen by now. Google with all their learning and training data, still can't correctly find and smudge license plates or faces correctly on Google maps. If that much processing power cannot correctly identify two classes of objects, what chance do these cars have to classify tons more objects + adapt how they steer based on that information in real time?
- Sublime
- GitHub.com
- ACM Digital Library
(The latter two are subscriptions.)
Things I've payed for in the past that I no longer use:
- MS Visual C++
- Omicron Pascal
- Application Systems Modula-2
- Atari ST GFA BASIC 2.0
- Berkeley YACC and FLEX port to TOS/GEM
- ...
Overall, many dev tools are free nowadays, which creates an expectation, perhaps, that it should all be free (I disagree in principle, but of course it is nice to see this trend progressing).
I appreciate that Kite is posting a post mortem for others to learn, and I wish they had been able to find a niche where people pay for their work. I love software tools as a work product, but have been told by many experienced people it's not a good area for making money.
The problem with Kite seems that their engineered first ("This machine learning AI is so cool, what can we do with it?" "I am a VC, are you doing AI? TAKE MY $$$$!") and only after burning through millions started to look at how to actually make money out of it.
And discovered that:
a) Hobbyist/individual developers rarely want to pay yet another subscription (can't justify it if you aren't making money with it & even $10/month subscriptions do add up!)
b) Corporate developers don't have purchasing authority. Everything must get approved, by both accountants and legal/compliance. Expecting a large company to pay a huge monthly/annual subscription fee for what is essentially a better autocompleter? Good luck with that.
That "Oh but your developers will be 18% faster!" argument is BS. 90% of the corporate developer's time isn't spent on typing code but on debugging, design, maintenance and meetings. Kite (or Copilot) don't help with that.
c) What about copyright/compliance issues? This has been trained on Github repositories - i.e. the same as Github's Copilot. How do I know where does the completed code come from? What about licenses on that code? Can I filter only for permissive/non-contagious (i.e. non-GPL) licenses? What about my code/whatever I type? Does it get sent to your servers? That alone is a complete no-go for companies.
In other words, a classic case where one shouldn't ask whether something could be done but whether it should. Someone outside of their engineering bubble and with a bit of business acumen would have told them that. Or at least told them to do a market research first, before spending all that time and money.
But hey, they had a good ride for the VC's money and are winding it down in an organized manner, without leaving a ton of shattered lives and a mountain of debt behind. So that's a plus.
This is the one-sentence summary about why the business failed, but it's kind of a strange way of putting it.
I am dead sure that there were plenty of advisers along the way who told the company's executives that its monetization plan was weak and unlikely to succeed. But everyone assumes that they'll be the exceptional case.
"It took too long to figure that out" makes it seem like the most likely scenario wasn't staring them in the face the whole time.
From what I remember, people got super annoyed at Kite for placing ads in open source projects and they just never caught on.
I see CoPilot all around me and it's generally well regarded and pretty widely adopted given how new it is.
Is there any data for this statement you can share?
(Thanks for working on kite and good luck!)
Credit to where credit is due. I worked in R&D here in a group tackling it for almost a decade ("program synthesis"), and while Copilot has a lot more to do, it solved so much of the usability & basic use case gap of what the R&D community had been attempting for years. Large language models & transformer models have been out for years, and the Github team executed well on adapting them.
(Separately: There _is_ an interesting question whether this space is VC-investable -- how likely will at least 1 startup here make it to 9-10 figures of revenue. But that's another story.)
It's not new, but the requirement to pay for it is.
Even plain old Jedi was a decent competitor to Kite.
So you we're no beat by billion-dollar CoPilot I'm afraid...
True, AI assisted coding does not deliver 10x. But as a user of another AI assistant, I feel that it gives me ~1.25x to ~2x improvement for the keyboard typing when I code. And that is respectable too :) AI for me currently allows me to tab complete some things that previously an IDE on its own was not able to.
Depending on how local he's talking, this isn't really true of Copilot. In my experience it will use context all the way up to the top of the file, even in very long files. And at least the Rust version even seems to look at the imports—if you have a use declaration it will actually correctly build and use structs in other files regardless of whether you've yet used them in the current file.
My understanding is that copilot is largely a self-supervised approach. They feed a massive body of (somewhat noisy) code into the model. The model really does learn a lot of structure on its own and this is a testament to deep learning on noisy datasets.
I'm guessing the "hooks" that they have already from IDE's, language-servers, etc. _are_ quite "structure-aware" - so they want the predicted structure as well as the code, so they can improve the typing experience beyond line-completion.
I think the estimation of 100 million for such a task is maybe too high? I don't know - it feels like you could actually get quite close to such a system by simply using thousands of custom prompt engineering tricks that prepend structure examples to the prompt?
From an old programmer perspective, it doesn’t make much sense, but a new programmer will not want to learn the old way, which will be effectively obsolete from lack of updates. If there’s any value to be derived from it, perhaps it is demand for hardware that will run enormously-inefficient code. The way that now you see people doing full sorts to get the third-largest value just because it’s easier to write it that way, you will see code that also does analytics and builds a distributed hash table to accomplish the same task, just because more capability means more usage means more suggestions to carry along that code.
I think it was a mistake to think of computer programs as a linear text language, but I don’t see this turning back. At some point, the concept of programming a machine will merge entirely with the method of interacting with a machine, which is to say, communicating intent, and then I suppose we can relax into a very comfortable full-service 5-star extinction.
You need to have 500 users to understand that, not 500K. A well-written postmortem otherwise.
When I read “non-local context,” it really drove home for me just how off the mark they were and changed the whole tone.
It also makes me think were they just hoping the solution would fall out of the sky? Seems irresponsible if that was part of their calculus.
This seems to be very much the standard story for "AI"; not quite there yet. Given the history, it's, er, surprising that people are constantly surprised by this.
I wish more projects would do this
It's also demoralising to see an entire category form without you, especially when you were working tirelessly towards it early on. I've really learned this the hard way also. Good luck to the Kite team in their future endeavours.
Are there a lot of businesses where individual developer productivity, with a narrow definition of LOC per hour, is the bottleneck?
I've worked for 10 years as a web dev and the bottleneck is very often at the product management level (tickets not ready, goals changing, haven't got the credentials for the 3rd party API yet..) and a minority of the time it's my brain (yes sometimes I need to think before I write code). It's rarely how fast I can write a function. So if you make me 18% faster at something I do 1% of the time... good luck making money out of me
Anecdotally, this was my experience at many companies before working at FAANG. But it's not my experience now.
They got wiped out by microsoft, github copilot, and litigation issues around AI provided code.
I was never aware they changed the architecture to keep code analysis entirely local. I would have purchased a subscription, had I known.
I agree that Kite didn’t deliver the 10x. I was an early user and tried hard to use it but didn’t find the benefit compelling enough to drop into my workflows, but it was very exciting.
I’m sure I speak for all of HackerNews when I wish you the best for whatever is next for the team.
Also, what are you good folks doing next?
I’m an individual developer and I pay for tools all the time. They just have to be of value to me. If developers weren’t paying for your tool, maybe look within.
I can't think of any dev tools I've paid for in this century; it seems better to rely on open source infrastructure.
Maybe they should have spent more budgets on marketing.
That said, I agree that no one wants to pay for developer productivity. The only exceptions are IDE and databases.
Anyone who's either worked at a developer tooling company or tried to sell to developers themselves (I personally did both, having worked at CircleCI in the past and now building my own developer tooling product at https://reflame.app, Show HN launch thread here: https://news.ycombinator.com/item?id=33134059) can probably back the observation that we individual developers are notoriously reluctant to open our wallets, even for products that we love and use daily, despite our high disposable income relative to professionals in other markets.
Gonna share a few of my own hypotheses for some of the contributing factors as comments below for discussion.
Would be fun to see folks share their own! Especially if you've seen successful strategies for how someone might be able to overcome these hurdles to selling products to individual developers at scale (a topic near and dear to my heart these days)!
Again, I'm totally guilty of this myself, since Reflame started as a side project initially to scratch my own itch, and I can't claim to have done an exhaustive search on the problem space before I started.
This results in a vicious cycle where every product, however innovative it might be at its inception, gets quickly commoditized by dozens of similar products immediately following any signs of traction, so they end up having to shift to competing on price eventually.
Combined with https://news.ycombinator.com/item?id=33691132 means any product that isn't available for free then eventually rots into obscurity due to the unfair distribution advantage of "free" in this market. Thus they are forced to offer a free version themselves and the cycle continues.
I certainly suffered from this myself to a rather extreme degree in the past, having categorically refused to pay a single cent for anything I used to build side projects with, until I started to seriously think about pricing for my own product. Eventually I realized throwing money at almost any problem where it could buy me more free time should be a no-brainer considering how highly I value my free time.
Tangentially, I think there's an interesting analogue in here to what Steam did in the PC games market, but I digress...
Most SaaS are priced completely independently of infra costs (or any other costs really), but we are much less likely to accept products priced with high margins on top of obviously low infra costs, even though that doesn't represent nearly the full cost of running a SaaS (which consists mostly of payroll due to how high our salaries typically are haha...).
We also like to justify this line of thinking by the argument that "well, I can build this myself in an afternoon" (significantly underestimating the real ongoing time investment required to build and maintain a SaaS product, even seemingly trivial ones) or "I can write some bash scripts and put this on the VPS that I'm already running anyways" (undervaluing our own time).
I've recently noticed a couple of companies open sourcing their product upon discontinuing the company -- is this a new trend or has it happened for a long time and I am only just recently noticing it?
> Then, our product failed to generate revenue. Our 500k developers would not pay to use it.
Isn’t it better to work with a smaller number of users but more closely first. Otherwise you burn the chance to impress all of those users. Plus with 100 users you have a decent sample but you can also reasonably interview them all one on one.
Now whether or not that's due to the fact that copilot had the financial resources to train a significantly superior ML model is another question, but throwing shade at copilot is a fairly transparent move.
I highly doubt that you failed! You blazed a trail forward for people in the future to follow. Financial success is not the same thing as taking a super tough problem to solve and then making inroads solving or starting to solve the many sub-problems (and their sub-problems) that invariably show up as a result of taking that path.
>"Then we grew our user base. We executed very well here, and grew our user base to 500,000 monthly-active developers, with almost zero marketing spend."
That's extremely impressive in my book! (By comparison, I failed to get 2 users -- for one of the apps I built -- and that was with marketing spend! <g>)
>"Then, our product failed to generate revenue. Our 500k developers would not pay to use it."
You might mean that there may have been an issue with communicating the VALUE of your product such that users would "see" (magical word, "see" -- "percieve", "understand", "observe in a way that you do") the VALUE of it -- such that they would be willing to equally-and-oppositely exchange their money for that VALUE...
Finally:
I do not think that you failed, and you have no reason to apologize to your investors, customers, employees and others.
You pushed the envelope -- and you created great value for future generations who will no doubt benefit from your pioneering steps in this gargantuan undertaking.
Well done -- and I think more people should appreciate you for that!
That being said, glad to see a lack of Our Incredible Journey type language here and more of a true postmortem of their business and technical decisions. It is rare to see a company go into so much detail when shutting down.
* Our diagnosis is that individual developers do not pay for OUR tool.
What sounds more plausible you:
- engineering managers hate free money
- it’s obvious to everyone that this statistic is bullshit
Sure, your people were great but they didn't innovate enough to make an attractive product (granted, AI code autocompletion is hard - I doubt we'll get something I'd be happy to pay before we reach GAI and we'll be all out of a job by then).
Oh and the "It's not the tech fault which is amazing, it's just a sales pipeline issue!"
Look, I understand caring about your employees and I said the same BS when my company failed trying to shift all the blame on me and not on my team. When you are in a startup it's everyone's job to say "hey, btw, what we want to do will suck because the tech is not there".
If you see something raise it and try to pivot, or you'll be out of a job with worthless grades.ß in
Maybe you could have cut your losses earlier on.
Many people I know find copilot extremely helpful. I think tools like it are about to become extremely important to the productivity of everyday developers. I seriously doubt it will take $100M to develop. The company Kite might have needed $100M to get there, but I bet you a few smart people working evenings in their garages can get there too.
Also the "nobody pays for dev tools" line is pretty obviously a weak excuse. Github is a developer tool that was worth $7B+. The truth was they just didn't provide _enough_ value to get people to pay for it. That's clearly true, and goes along with their idea that they were too early. Not that the problem is impossible.
I am very impressed and happy to see the open-sourcing of their code like this. I often find myself thinking about how much human knowledge and effort disappears when a company shutters and all of their documents, code, etc go with them.
Sadly, I never heard of Kite until copilot came out. As someone who pays for tools, I would have considered it (have paid for JB for years, various atlassian tools, and other utilities/etc).
Whoever wrote this will go far.
Quite simply developers are not decision makers. Often engineering managers aren't even decision makers. I fully believe if you want to dominate in that area you need to target decision makers who force it upon their developers. How many of us have been told we're using X database or we're using X project management tool or even X virtualisation system? Management makes these decisions which is why if you go an AWS conference you'll find majority of the people there aren't techies but management and lots of the talks are aimed at management understanding the tech.
I've seen many other commenters lament the fact that "even though devs make a boatload of money, they don't want to pay for their tools".
This may be true.
But I think the biggest issue is that most developers are only developers "at work". And I've seen far too many people work with subpar tools (old, dingy PCs that would take ages to do anything). Management thinks this is fine, and that a 5 yo intel U laptop with 8 GB RAM is AOK for running heavy computations in 2022.
So, going out of their way to buy some "text editor" when vscode is free? Not gonna happen. Even if the devs themselves are convinced.
Not to sound overly mean but it might have been a good idea to start with testing this idea first/earlier. Additionally, it seems to me like they didn't do a great job at identifying their customer. It's probably not individual devs but rather the people they work for. So you're in a B2B business and need to sell it that way.
The meaner response would be that it seems like developers do not pay for YOUR tools. Seems like there are plenty of paying customers for copilot for example.
What?
You're supposed to create the technology, not wait for others to create it. That's why VCs give you money, isn't it?
> We built the most-advanced AI for helping developers at the time, but it fell short of the 10× improvement required to break through because the state of the art for ML on code is not good enough.
Aren't you supposed to advance the state of the art?
> but the problem is very engineering intensive
So you weren't a technology company?
Disagree. I pay for Visual Assist as an individual because its a huge productivity booster. I suspect the issue is pricing vs perceived value.
> It includes our data-driven Python type inference engine
I couldn't find which repository this lived in. I am very interested in it, as my team maintains a few open source static analysis and code generation tools. We'd be interested in trying this out.
Copilot ended up being rarely helpful for me. But on the other hand, MS IntelliCode (I think it only works for C# in full Visual Studio) was a fantastic productivity tool that actually sped up writing code because it does understand the structure of C# and your codebase. Wish it was available for other languages and VSC
Counterpoint:
Maybe instead of blaming potential customers for not finding enough value for your product, you might need to start looking inward.
I gladly paid for my own personal license for R# that I kept across four jobs over 8 years. I only stopped paying for it because I no longer develop in C#.
My favourite use is at the command line. It's great!
I pay for it myself and use it in all sorts of contexts.
EDIT: Actually, perhaps that is actually smart. If you want to find people who would pay for dev software you probably should target people who pay for dev software already. Jetbrains is better for paid plugins than VS Code by this logic.
> We failed to build a business because our product did not monetize, and it took too long to figure that out.
Yet people always defend telemetry in software, saying it's how they improve their product. 7 years of telemetry, and they couldn't figure it out?!
A lot of work I think and as is pointed out here devs wants their tooling for free.
- developers willing to use AI crutches instead of their own brains to write their code;
- developers unwilling to pay other developers while being paid themselves by companies whose profit models are often far removed from the honest craft of developing something wholesome.
Not sure about Kite though
Hopefully the next project goes well!
They do pay for tools, but not enough to make it a full time. I've got a few "side" projects that bring in a few K/month each.
I read about and tried evaluating Kite at the time and it seemed like it was in some kind of private invite stage. I remember thinking it must have been acquired and wasn’t taking new users. This must have been an incorrect take.
I’m surprised Tabnine is not mentioned in this thread at all, though because that was acquired and afaik is still operating.
Before copilot came along, Tabnine, not Kite, seemed like the ai took to beat.
I also remember a Python dev relations person from Jetbrains going on a podcast and clowning on AI code completion. That was in April of 2021. [2] A month later copilot dropped.
The very strange thing about that was Jetbrains described efforts to build an ML-based code completion plug-in in 2016! [3] It obviously failed to follow through on that.
I still think G Co pilot represents a threat to jetbrains IDEs overall. Even the packaged autocomplete can’t compete on basic stuff copilot does now.
I disagree with the idea that AI code completion is not good enough yet. I see that said all the time and yet it can masterfully fill in boiler plate today.
It can be way better, particularly in languages outside JavaScript and Python, but it’s usable now and maybe even profitable as a service if the business is not leveraged by VC capital.
If you listen to the September interview with Eddie Aftandilian of Github Copilot you would realize how early it still is for that product, as how to measure success in code completion is something still requiring behavioral patterns that are still being recorded.
Here’s the episode, listen 20 mins in: https://www.se-radio.net/2022/10/episode-533-eddie-aftandili...
[1] https://news.ycombinator.com/item?id=25074393
[2] https://twitter.com/jetsetter/status/1379438096232587265
[3] https://blog.jetbrains.com/idea/2016/09/share-your-stats-to-...
copilot: "Get code suggestions in more than a dozen coding languages including Python, JavaScript, TypeScript, Go, and Ruby", how about c, c++ even lua here? if they cover c and c++ I can pay $10 per month right away.
Builders buy hammers, drills. We should be parting with our money for tools that multiply our earning capacity as contractors and consultants.
I would pay 5x as much for many of the tools I buy too, such is their value.
Is it easier to build AI for pure functional programming languages?
Does it mean employees don't pay for the tools? Or that single-person ("individual") independent developers don't?
Why wouldn't a developer pay $100 for a tool that saves a day of work for them?
They are clearly useful but people still don't want to pay.
Good riddance to bad rubbish
First, we failed to deliver our vision of AI-assisted programming because we were 10+ years too early to market, i.e. the tech is not ready yet.
We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough. You can see this in Github Copilot, which is built by Github in collaboration with Open AI. As of late 2022, Copilot has a number of issues preventing it from being widely adopted.
The largest issue is that state-of-the-art models don’t understand the structure of code, such as non-local context. We made some progress towards better models for code, but the problem is very engineering intensive. It may cost over $100 million to build a production-quality tool capable of synthesizing code reliably, and nobody has tried that quite yet.
Nonetheless, we could have built a successful business without 10×’ing developer productivity using AI, and we did not do that.
We failed to build a business because our product did not monetize, and it took too long to figure that out."
That's not the same thing as being too early to the market. That simply means you didn't have a solution capable of solving a problem.
lol
$100 million is nothing tbh.