This announcement comes after a 4 month sabbatical where Karpathy said he wanted to take some time off to “sharpen my technical edge,” which makes it sound like this is the result of frustration with the technical approach instead of burnout.
The fundamental flaws are in the decision-making being based upon 10-30 second feature memory, ignoring features outright, and only depending on visible road features instead of persisted map data.
For instance, near my house there's an intersection where it will try to use a turn-only lane with a red arrow when it's trying to go straight thru a light. 100% of the time. Even if I'm in the correct lane with no traffic around.
That's because the turn arrow on the ground is worn off. It is not a perception problem, it is
a) ignoring the obvious red turn arrow signal's significance for lane selection deliberately b) makes no attempt to persist or consult map data for 'which lanes go where' c) it completely disregards painted lines on the ground in the "no drive here" striping.
Also one block from my house, FSD will stay still indefinitely waiting for trash cans (displayed as trash cans) to clear the leftmost lane so that it will turn left.
None of the failures I encounter are due to lack of perception.
I think that's optimistic. Ten plus years.
There's too many exceptions. In my home town, an intersection. Imagine, if you will:
Traveling westbound there's four straight ahead lanes. There's a traffic light for each straight lane. The traffic lights will alternate "left two straight, green; right two straight red" and then "left two straight, red; right two, green". It does this because there's a tunnel and a roundabout right there. I guarantee that FSD will choke on this.
Tesla has been collecting thousands of dollars, each, from car buyers and utterly failing to deliver what it represented, and keeping the money year after year. Would be let GM, Toyota, or Audi do this? Where is the criminal prosecution? Where are the refunds?
I vividly remember a conversation I had with an acquaintance who had recently taken an engineering position with an AV company that occurred circa late 2018. He claimed that they were at most 1 year away. In fact, his exact words were something along the lines of "They're already here. It's just a few edge cases to work out and some regulatory hurdles to overcome."
The reality is that the first 80% of the problem had been solved quickly and significant progress had been made at the time on the next 10%. The end was in sight. Unfortunately, that next 10% ended up taking as long as the first 80% to solve, and the final 10% will likely take decades if it's even possible.
You aren’t beta testing complex automation system that’s operating on public roads. To do any meaningful testing you should have defined operation domain, specific behaviors to test, direct line to the engineering team to report issues, etc, etc.
Elon has been over-promising(i.e. flat out lying) about self-driving every year since.. 2014(there's a youtube video compilation of it)?
It seems like his strategy is to just come up with increasingly grandiose promises every year when he fails to deliver on his past promises. He's trapped in his swirling vortex of bullshit. Very worrying to see Karpathy leaving...
Steve Jobs was also known for his “reality distortion field”, so maybe it comes with the territory.
That said, FSD progress seems asymptotic and the Optimus thing always seemed like bullshit.
Elon in 2026: “by 2028 we’ll have FTL drives”
Elon in 2028: “time machine!”
They are just going about it better but not trying to selling it.
Any reason why everyone seems to be stuck on this problem?
"A Cruise autonomous vehicle ("Cruise AV") operating in driverless autonomous mode, was traveling eastbound on Geary Boulevard toward the intersection with Spruce Street. As it approached the intersection, the Cruise AV entered the left hand turn lane, turned the left turn signal on, and initiated a left turn on a green light onto Spruce Street. At the same time, a Toyota Prius traveling westbound in the rightmost bus and turn lane of Geary Boulevard approached the intersection in the right turn lane. The Toyota Prius was traveling approximately 40 mph in a 25 mph speed zone. The Cruise AV came to a stop before fully completing its turn onto Spruce Street due to the oncoming Toyota Prius, and the Toyota Prius entered the intersection traveling straight from the turn lane instead of turning. Shortly thereafter, the Toyota Prius made contact with the rear passenger side of the Cruise AV. The impact caused damage to the right rear door, panel, and wheel of the Cruise AV. Police and Emergency Medical Services were called to the scene, and a police report was filed. The Cruise AV was towed from the scene. Occupants of both vehicles received medical treatment for allegedly minor injuries."
Now, this shows the strengths and weaknesses of the system. The Cruise vehicle was making a left turn from Geary onto Spruce. Eastbound Geary at this point has a dedicated left turn lane cut out of a grass median, two through lanes, a right turn bus/taxi lane, and a bus stop lane. It detected cross traffic that shouldn't have been in that lane and was going too fast. So it stopped, and was hit.
It did not take evasive action, which might have worked. Or it might have made the situation worse. By not doing so, it did the legally correct thing. The other driver will be blamed for this. But it may not have done the thing most likely to avoid an accident. This is the real version of the trolley problem.
[1] https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
[2] https://earth.google.com/web/@37.78169591,-122.45337171
[3] https://patch.com/california/san-francisco/speed-limit-lower...
To solve the self-driving problem we need "smart" A.I., which means we have to approach it with systematic engineering, and the solution will probably involve some combination of better sensors, introspectable neural nets, symbolic A.I., and logical A.I.
Because it's really, really difficult. A lot of AI-ish stuff pretty rapidly gets to the point where it _looks_ quite impressive, but struggles to make the jump to actual feasibility. Like, there were convincing demos of voice recognition in the mid-90s. You could buy software to transcribe voice on your home computer, and people did. And, now, well, it's better than in the mid-90s certainly, but you wouldn't trust it to write a transcript, not of anything important. Maybe in 2040 we'll have voice recognition that can produce a perfect transcript, and human transcription will be a quaint old-fashioned concept. But I wouldn't like to bet on it, honestly.
And voice recognition is arguably a far, far easier problem.
ML maximalism focused on the narrow problem of 'solving driving' while not recognizing that any task as complex as driving requires probably something closer to general intelligence, and theoretically the field has been impoverished in favor of "throw more graphics cards at everything".
We can't build a robot which can walk down a sidewalk without running into people either. The sensor tech and mapping fidelity are red herrings. People drive well because only people are good at predicting human behavior.
I certainly wouldn't argue with you that it isn't ready for prime time and wide distribution, but it is interesting to see their progress in San Francisco, a much different driving problem.
If it takes them 10 years to get to prod in Mesa, two (maybe three?) in SF, maybe they start shrinking that a lot in metros without winters. ¯\_(ツ)_/¯
I thought they had real self-driving taxis in Pheonix that you can order? Real ones, with no safety driver.
That definitely sounds "better", even if it is heavily geo-fenced.
Because they're all trying visual- or line-of-sight methods only, I call this the "robo-human" fallacy in ML: trying to automate the processes that humans undergo so that you eventually have a drop-in replacement for a human. But that is a myopic and unimaginative approach because you could be re-assessing the system itself and eliminating inefficiencies that lead to poor performance.
In the autonomous vehicles space, there is massive potential for self-organizing swarm algorithms to control pelotons of cars, rather than individual cars with no intrinsic sense of the general flow of traffic. You wouldn't need a top-down "commander" style architecture, it could be designed so that cars only talk to their immediate neighbors and emergent patterns keep traffic flowing smooth and fast.
I have always been skeptical of the attempts to reduce the amount of information about the road that a car receives. (Moving from stereoscopic to monocular vision to save the cost of one camera seems just stupid.) But people who dream of "smart cities" really seem to see little more than The Jetsons in their mind, and it limits the scope of research to our detriment.
How is AI supposed to confidently distinguish a real stop sign from someone/something holding up a picture of a stop sign?
Yes, this is a weird edge case, but I think it gets at the core issue being that it takes way more sophistication to release this tech into the wild then ppl would like to admit.
Tesla is taking a fundamentally more broad and deep approach - working with the fundamental fact that a pair of visual sensors and a compute engine (eyes & brain) can successfully figure out driving in strange areas in real time, ergo, it should be possible without a map/model or lidar. Once they get it solved, it will be solved once and for all. Bigger gamble, bigger payoff. Equipping the car with dozens of eyes is the easy part. The question is whether enough compute power can be brought to bear on solving the recognition problems, and the edge cases. They have obvious issues with failing to recognize large objects like trucks in unexpected orientations, left turns etc. Using millions of miles of live human driver data as a training set is great, except that the average driver is really bad, so it's entirely polluted with bad examples, ESPECIALLY around the edge cases that get people killed. There, examples from professionally trained drivers, who really understand the physics and limits of the car, adhesion, traffic dynamics, etc, are what you want to train on, but that isn't what they have. It is also possible that even if the set of training data would actually be sufficient, the big question will kill them - perhaps the solution requires orders of magnitude more compute power to approach human performance, and they just don't have the hardware to simulate human compute power. So, have they just hit the limits of what their compute power can do?
I think Tesla's approach is fundamentally the way to go, as it is a general solution, compared to everyone else's limited map/model approach.
But both may require either or both a more specifically programmed higher-level behaviors, and/or something much closer to AGI than exists, something that has actual understanding of the machine-learned objects and relationships, which does not yet exist (if one is known, pleas correct me - I'd love to know about it).
They have a big head start, but other car companies are now investing much more in battery tech etc and will quickly catch up. Not to mention Tesla's have terrible build quality, they have a lot of shady business practices like overcounting sales, reusing sold parts etc which came out in the recent leak.
Thing like the 4860 battery which were so hyped turn out to be not that much better. FSD is years away. Stop selling vaporware.
What they need to focus on is things they innovated on like OTA updates, integrated systems, no dealerships etc.
They have terrible manufacturing quality. The screens melt in Arizona heat. Maybe the ride is cool and feels good, but the car itself is not incredible.
They may want to think about that strategy soon. Model 3 is starting to seem dated (not to mention Model S, which is ten years old). There are very competitive alternatives on the market now that have strengths where Tesla is weak, and which are not especially weak in the areas Tesla is strong.
How so? They're not selling robotaxis or building factories to build them
> Tesla has repeatedly promised FSD is right around the corner
Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist" or whatever standard feature from auto manufacturers they've renamed
Because they chose to back themselves into that corner. Musk says that Tesla is worth nothing without full self-driving. Certainly it's the only thing left to justify the stock price:
https://electrek.co/2022/06/15/elon-musk-solving-self-drivin...
> Which means it's years away and/or "FSD" means "automatic cruise control and lane keep assist"
Well, more precisely it means Musk has been lying about it for nine years straight:
https://jalopnik.com/elon-musk-promises-full-self-driving-ne...
The lies have been profitable so far. People have bought into the false promises. Perhaps they'll start demanding refunds for the full self-driving they paid for that has still not been delivered.
That's exactly what I would expect someone burning out to say. You feel the burnout so you need time to get over it and feel 100% (regain your technical edge). You're still burnt out after 4 months, so you don't come back.
Frustration with the technical approach can also cause burnout.
There is usually a hierarchy of sensors, mainly for redundancy. Example: Bumper sensory at the wheel base, sonar / Lidar at the mid, and a camera at the top for advanced sensing.
For the sake of cost cutting Tesla has done away with their radar sensors at the front of the vehicle. It would be a substantial cost overhead, but have very real repercussions when it comes to safety, while also providing a "ground truth" to what at least the front facing cameras are seeing.
I don't think Lidar is a practical sensor for them to adopt, because it is quite bulky and has limited viewing angles, but I would expect them to have adopted some novel, lower cost radar solution.
Apart from the lower cost of the camera, I think Elon's rationale for having a camera only FSD is not valid, has made the problem needlessly complex and unsafe. He believes since we have eyes, and we can drive a car, then it should be sufficient to drive the car, but we only use eyes because these are the sensors we were born with, it is the best we have. In my mind, Elon's approach is like looking at a horse, and saying to yourself, that you want to build a car based on a horse, where instead of wheels, you have four mechanical legs, and those mechanical legs are limited is so many ways, but they should still at least "work", but there is no reason to limit locomotion in that way. The same with the vision system on a FSD, the whole spectrum of light is available, with any number of configurations, providing data at rates and with precision far beyond what a camera system can do.
My background is in physics, but I find myself having a growing appreciate for the vision-only stack. It's really challenging building a formal understanding of the world that is robust to outliers that are so numerous as navigating in an urban environment. With vision, you have multiple kinds of information that are highly correlated (colour, spatial distribution, depth, etc) that are self-consistent. Whereas, fusing radar with vision, where object responses to radar are highly geometry & material dependent, is a much harder task.
I'm really not an expert, so this reads more as an opinion than an experienced view, but I can see the merits in doubling down on vision.
And Tesla cars have more than one camera on them. The front-facing camera is actually an array of 3 cameras (the two farthest ones are at about human eyes distance), but they're also equipped with forward and rearward looking side cameras, and back cameras.
I think Tesla underestimated how hard vision-only FSD is, but having a single camera (they don't) is not the reason.
LOL no, he was jumping ship already.
BTW, Andrej, if you're reading this, it is not just excellent it is beyond excellent. I do a lot of tinkering with transformers and other models lately, and base them all on minGPT. My fork is now growing into a kind of monorepo for deep learning experimentation, though lately it started looking like a repo of Theseus, and the boat is not as simple anymore :)
Well, I'm not sure that anyone's tech stack is capable of solving it; the live examples of robotaxis are, well, not something you'd bet your company on (and generally their creators are _not_ betting their companies on them). There was, I think, a decade ago the idea that fully self-driving cars were a near-term inevitability. That's fading, now.
Both clauses seem wrong.
How so?
If humans can master driving with 2 eyes looking forward, why would a car with plenty of cameras in all directions not have sufficient sensory input to master it?
The problem is the software, not the sensors.
you're using elon's own argument btw, are you repeating that knowingly
I think Karpathy realized (probably way back) that cheap sensors + no HD maps + their (reckless) public testing feedback loop doesn't advance towards L5 self driving and is bailing out. Karpathy has always backed Elon Musk whenever he talks about their technical approach, so it can't be frustration with the approach all of a sudden.
Tesla filed with the FCC in May to get authorization for a new radar system.
Does anybody seriously think Karpathy would step down if FSD was really close to be released ??
It really starts to feel like Tesla is a huge fraud which is about to be uncovered.
In fairness, this is just another in the long line of ridiculous things that he is prone to saying.
The tech is all right, and I got to try auto pilot at a discount. Unfortunately the phantom braking made AP completely useless with passengers who would freak out and complain. However, when it worked it was quite nice but I ended up using it way less than I'd hoped. Glad I didn't pay 12k for it!
The best part of owning the car was the insane acceleration and supercharger network at the beginning. But, that got annoying as more people started getting Teslas. Going on longer trips meant a ton of anxiety especially since some superchargers in cities would be packed. Worse, some would be out of order or slow charging. After the gimmick wore off, wasting 45 minutes to go another 2-3 hours started becoming annoying. And before someone asks why 2-3 hours, its called hills. California is full of them, and especially where I live I lose so much efficiency climbing hills.
Anyway overall I'd rate the car 5/10. Fastest car I've ever owned. Beyond that it was pretty much exactly as they described - a beta product. I'll probably try a Tesla again in 5-10 years.
They'll get eclipsed by other electric car manufacturers real quick.
Edit: more specifically, the parts break and they are difficult to replace. The battery degrades. They stopped providing maps to the vehicle unless I'm willing to spend several hundred dollars to replace the media console, they've told me I'm covered by a recall/warranty but have been unable to schedule the appointment.
It's a real shame because in principle having a manufacturing facility making great electric cars in the bay area would be a real win. Musk's reality distortion field is cracking.
As for luxury, quality, ride comfort - they’re just ok
What's shocking is how many people interpreted it literally--that the value is literally zero without self-driving--as if the successful EV business is in fact unsuccessful.
Like, at least give one metric on which they are measurably “the best”.
But are the financials of the company also real ? The prospects of future products ? Robotaxis, FSD, Cybertruck, Semi ?
But in itself, just making on of the best electric cars today would justify a valuation of 1/10 of what Tesla currently have.
I still admire Musk and Tesla for having started the electric revolution. But by 2025 (and maybe already are), they will just be one of many electric car manufacturers - somewhere in the middle of the pack.
Life is not only about work. I stepped down from the company I funded after 14 years, it never stopped growing after that. Some people just get bored after doing one thing for a long time and want to explore other areas, especially if they always had broad interests.
> It really starts to feel like Tesla is a huge fraud which is about to be uncovered.
I seriously doubt that my Model S will somehow stop working so well and turn out to be just a fake car after 5 years.
I say that I must be imagining that I’ve actually been driving an EV for the past four years.
I must’ve dreamt having to drive only 2 hours out of a 12 hour road trip.
Bugs? Yea. Some really dangerous ones we all have read about. Manufacturing defects? Yup! Missed Deadlines? Hell yeah.
But a fraud? Nope.
It was always the case.
Why do people leave companies in this manner?
1) It sometimes can be hard to leave a company when you are "in the thick of it." A sabbatical can give you personal time to reflect on whether you want to stay or not.
2) Sometimes people use sabbaticals to prep/perform job interviews or plan career transitions.
3) Sabbaticals can allow you to quit early while waiting for vesting restricted stock units, employee stock plan sales, retirement contributions (matches), etc. There are certainly many more timed bonuses available for senior leaders.
Employer: Are you sure? Why don’t you take some time off and think about it?
Not only that, but Tesla has played the Innovator's Dilemma game from the position of the upstart financially, but targeted the segment of the market that incumbents will defend to the death (luxury cars).
Tesla could have gone a different way and played the game from the true upstart: targeting the low end of the car market. Attack from below. But it didn't do that.
Incumbents always win at the sustaining innovation game. The electric power train is a sustaining innovation for the automobile industry. It doesn't break any incumbent's business model (financing the purchase of expensive cars), especially at this point. And we're now seeing this with all of the EV introductions and announcements from incumbents. Oddly, though, there are plenty of upstarts trying to do exactly what Tesla tried - attacking the blubber-rich end of the market with an immature technology.
The fall can cascade quickly, as the incumbents are stuffed with debt. When sales of ICE fall due to the growing EV segment they will have a hard time seeking the funding necessary to transition. IMO it's all a bit late. But good luck to them. From my observation only Ford and VW really appreciate the situation they are in, and they are trying hard to navigate out. Hopefully they survive. GM is screwed.
That doesn't make any sense. "Legacy auto" knows how to make cars. To their final assembly factories there's not a huge difference between a BEV, PHEV, or an ICE drive train. So long as they feed in components they get cars out. They definitely know how to get components made to feed into their factories.
That's a place where they have an advantage over Tesla. They can make BEVs that break even or lose money because they have a whole line of ICE cars making a profit. Tesla only has their up market BEVs to make their money.
Tesla doesn't have a moat around BEVs. Now that "legacy auto" is making them Tesla is just another BEV manufacturer. As their market share erodes they're going to have a harder time maintaining their price premiums. They also don't have the deep bench of fleet sales that "legacy auto" has. The places they're trying to diversify (Power Wall, solar, etc) aren't markets that support the premium prices they currently enjoy with their cars.
My take is the opposite. Incumbents already have like 90% of what Tesla had or chose to build from scratch. EV's are not complicated at all compared to ICE, and incumbents have been refining their manufacturing process for decades in a brutally competitive industry.
Unless Tesla can somehow get an effective monopoly on battery tech or FSD, which they are not, they're going to be just another car company. Nothing they can deliver can't be done by others.
Personally I think it's way more likely Waymo or Apple deliver FSD first, with broad industry partnerships wiping out Tesla's advantage.
https://www.forbes.com/sites/neilwinton/2022/07/11/ev-makers...
This meshes pretty well with them not having anything in the compact, pickup truck, offroading/camping segments (luxury, or not). They also don't have any commercial vehicles like delivery vans.
All of those segments are being filled in rapidly by competitors. Also, the first real non-Tesla luxury sedans have been on the market for what, under a year?
Because there’s 0 chance they let go of the first, since that’s integral to any new car these days.
Unless I have been reading lies, this statement is at least half false
If you life in San Francisco, CA, you'll often see Cruise cars roaming the streets at night collecting data with no physical driver present.
The only thing that needs to be solved fully is highway with auto lane change. That's literally it. And the system needs to do eye tracking like Comma Ai. If your adaptive cruise control makes you put your hand on the wheel it's useless.
Just perfect highway driving first. Trucking alone would massively change if goods could be transported to edge of urban areas autonomously.
I'm sure it's being worked on too, but puzzling why that use case is not priority 1, 2 and 3
Data annotation can be cheaply outsourced or scaled up and down without much affecting progress on self-driving. Karpathy leaving says more about that progress to me.
Only if you don't care about quality. In my experience using annotation companies leads to a conflict of interest - annotate more to earn more, or annotate better.
Except he failed to achieve what he claimed he could.
I’m not sure if many people 5 years out of their PhD can say they did anything close to what Andrej did.
Once again, where are the robot-taxis as promised for release in 2020?
My point is, people leave their jobs for many reasons other than how well the company is doing. At least I hope so.
