HN Theater @HNTheaterMonth

They are reliable enough to use for FSD: https://youtu.be/hx7BXih7zx8 so i think it's reliable enough for park assist, summon etc. Yeah you're def right about the moisture sealing... it's been hit or miss on production tolerances

⬐ dagmx

> They are reliable enough to use for FSD

I disagree. regular navigate on autopilot and the FSD Beta are still very flakey outside of pristine conditions for me.

At this point, I either am expecting Tesla to go ahead with a camera upgrade with an expensive retrofit in the future, or FSD will never fully realize.

The actual report is here: https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_...

It failed to detect a stationary mannequin in one specific scenario. Does this generalize? Does this mean there's a major safety problem? These would be assumptions that this test can not prove.

A comprehensive safety analysis has to also include these scenarios: https://youtu.be/hx7BXih7zx8?t=205

⬐ asdajksah2123

It couldn't detect a stationary child sized mannequin from over a 100 yards away on a wide open road completely clear of distractions in 3 out of 3 tries.

I'm struggling to see how any generalization would not be a much worse scenario.

⬐ fallingknife

Cars don't have a stopping distance of 100 yards until they are going highway speeds. So that doesn't seem like a safety issue as children standing still in the middle of the interstate isn't really something that happens.

⬐ manishsharan

>> children standing still in the middle of the interstate isn't really something that happens

No. But there could be protesters blocking the highway. As annoying as they are, running them over is not an option.

⬐ mcguire

The cars in question hit the mannequin. The car was driving in FSD mode for 100 yrds before it hit the mannequin.

⬐ fallingknife

Ok that's different. The way the comment was phrased made it sound like the mannequin wasn't detected until it got to 100 yards distance.

⬐ hgomersall

I think the question is not "does the test generalise?", but "does FSD generalise?", which it emphatically does not.

⬐ jsight

It always ultimately comes back to safety. How many times has AEB avoided a pedestrian that was not detected by the driver of the vehicle? Is that greater or smaller than the number that were hit by drivers with a false sense of confidence in the software?

The Dawn Project test cannot answer such questions.

I believe that NHTSA is working on this and related questions across the industry and I expect them to produce a more useful result on their own than what can be produced by single-minded reports like this.

⬐ hgomersall

No, it's not a purely single variable optimisation. If this system (and let's be honest, we're just talking about Tesla here) cannot avoid hitting an object that every driver should easily see, then it's not fit for purpose. The reason being that people take action based on certain assumptions about what drivers will do. If a pedestrian steps into a road to cross when an oncoming car had plenty of space to both see and slow, then it's not ever acceptable to hit the pedestrian. This is a normal urban interaction.

⬐ ZeroGravitas

> demonstrated conclusively that the software does not avoid the child or even slow down when the child is in plain view.

I can't fugure out the "even slow down" claim when the car goes from 40 to 25mph by itself?

⬐ asdajksah2123

They are saying that it slowed down to 25mph as a consequence of being placed into FSD mode.

But it did not slow down anywhere near the mannequin, and even sped up a little bit when hitting the mannequin in 1 trial.

⬐ AareyBaba

Similar testing of Tesla on this channel https://youtu.be/p7lp5f0aqzU?t=74 shows it failing badly on simple hazard object scenarios like a chair, wooden pallet, trash can etc. Also when the car does give an alert, it is too late for a human to react.

> All evidence from their own AI/AP team and presentations is that there is no real design and system validation going on over there. They're flying by the seat of their pants, introducing potentially lethal regressions in every update.

What is this evidence?

I've seen a few talks from Andrej Karpathy that indicate to me a more deliberate approach.[0] "Software 2.0" itself seems like an approach meant to systematize the development, validation & testing of AI systems, hardly a seat-of-your-pants approach to releases. I have my own criticisms of their approach, but it seems there is pretty deliberate care taken when developing models.

[0] https://youtu.be/hx7BXih7zx8

⬐ kelnos

> What is this evidence?

I think the onus should be on Tesla to prove that their testing and validation methodology is sufficient. Until and unless they have done so, Autopilot should be completely disabled.

I really don't get why the regulatory environment is so behind here. None of these driver assistance technologies (from any manufacturer, not just Tesla) should be by default legal to put in a car.

⬐ phkahler

>> They're flying by the seat of their pants, introducing potentially lethal regressions in every update.

>> What is this evidence?

Without a documented development and testing program, every development is essentially this.

⬐ refulgentis

I see your point, to OP's point, I know a couple people who were horrified at what they saw and it did not match this public talk. Both started at least 6 months after this video, and both left Tesla within 8 months, of their own volition. Unfortunately, off the record.

Not to disparage Andrej, sometimes (frequently, even) what executive leadership thinks is going is not the day-to-day reality of the team.

⬐ plankers

can confirm, a former coworker had just come from Tesla 5 years ago and he had serious ethical problems with his work over there. Tesla is killing people through negligence and greed, it's pretty disgusting, but par for the course

⬐ stefan_

This is the Karpathy that gave a big talk about how vision was superior to radar when Tesla dropped all radar units at the height of the chip crisis. Now they are bringing radar back.

Give it a few years and they will probably introduce LIDAR.

⬐ pbronez

Tesla is bringing Radar back? First I've heard about it, and good news if true.

⬐ aaaaaaaaaaab

https://electrek.co/2022/06/08/tesla-files-use-new-radar-con...

⬐ epgui

Wasn’t this approval based on an application from 2018?

⬐ justapassenger

I’ve been working few years ago at a very big tech company, focusing on validation of the AI systems.

It’s all smoke and mirrors. You cannot perform proper validation of AI systems. Rollbacks of new versions of ML models are very common in production, and even after very extensive validation you can see that real life results are nothing like what tests have shown.

⬐ urthor

Can't you do outlier detection, and disable the AI if the input wasn't in the training set?

⬐ justapassenger

How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that?

You basically put another ML on top of ML, to correct it. I’ve seen that in use in production systems, and it helps with some problems and generates new ones. And if you thought that reasoning about correctness was hard before…

And what do you mean by disabling AI, if input wasn’t in the training set? That’s the whole point of ML, to reason about new data based on data seen in past.

⬐ urthor

> That’s the whole point of ML, to reason about new data based on data seen in past.

I think we like to think this is true.

In reality, I have seen a lot of real world ML models. I wouldn't trust ANY of them to do extrapolation. There are just tons of real world problems, and extrapolation is HARD.

I have to put extremely tight boundaries on ML models for deployment scenarios, and ALWAYS have a backup rule engine in case the ML model comes up with an answer that has a low confidence score.

> How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that?

> You need to write some rules that could look at it.

Pretty much. Any time ML is involved, you will need TONS of lines of code.

In short, tightly define the target classes your ML model deals with.

Any variable that falls outside your tightly bound list of target classes, you have to deal with using a rules engine. THEN you need to spend a lot of time doing work to minimize false positive classification in your target classes.

And make sure that "false positive, high confidence" classifications don't do racist things/lose the business a lot of money things.

ML projects are just a ton of work. You essentially make the entire ML workflow, and you NEED a backup "not-ML" workflow.

In my experience, 50-80% of normal software engineering projects fail.

90% of ML projects fail. Square the fraction of normal software projects.

ML is complex AND it's a ton of work. Really, really hard.

There is a mix of a "neural net planner" and a "explicit planning and control" in traditional code[1]. The explicit planner has the last word, and uses input from both the "vector space" and the neural net planner.

Karpathy has commented about the neural nets gradually replacing the traditional code. See his presentation[2] around 18:45.

[1] https://saneryee-studio.medium.com/deep-understanding-tesla-...

[2] https://youtu.be/hx7BXih7zx8

⬐ kirillzubovsky

#1 is a very good set of articles. thanks you!

You can learn about their systems by watching talks by Andrej Karpathy. As a robotics engineer interested in vision, their architecture is inspiring. This talk [1] is a good overview but each talk he gives is a little different so search for more if you want to know as much as possible.

But the big thing is that their autonomy computer can be programmed to look for odd scenarios and send them back home. Tesla uses their fleet of hundreds of thousands of cars to collect edge cases like this, and then they have a kind of compartmentalized neural network system that breaks apart disparate tasks. With their collected examples they can create unit tests to ensure that the moon stops activating the stoplight detector. Once trained, the unit tests presumably help ensure they don't end up with future regressions.

So basically every time you see a Tesla do a weird thing, there is a good chance it will stop doing it soon enough. At least if it's hitting hacker news.

[1] https://www.youtube.com/watch?v=hx7BXih7zx8

Not yet. I have a four wheel drive robot I designed with four 4k cameras feeding in to an Nvidia Jetson Xavier. [1]

Just getting it to navigate itself using vision would mean building a complex system with a lot of pieces (beyond the most basic demo anyway). You need separate neural nets doing all kinds of different tasks and you need a massive training system for it all. You can see how much work Tesla has had to do to get a robot to safely drive on public roads. [2]

From where I am sitting now, I think we are making good inroads on something like an "Imagenet moment" for robots. (Well, I should note that I am a robotics engineer but I mostly work on driver level software and hardware, not AI. Though I follow the research from the outside.)

It seems like a combination of transformers plus scale plus cross domain reasoning like CLIP [3] could begin to build a system that could mimic humans. I guess as good as transformers are we still haven't solved how to get them to learn for themselves, and that's probably a hard requirement for really being useful in the real world. Good work in RL happening there though.

Gosh, yeah, this is gonna take decades lol. Maybe we will have a spark that unites all this in one efficient system. Improving transformer efficiency and achieving big jumps in scale are a combo that will probably get interesting stuff solved. All the groundwork is a real slog.

[1] https://reboot.love/t/new-cameras-on-rover/277

[2] https://www.youtube.com/watch?v=hx7BXih7zx8

[3] https://openai.com/blog/clip/

⬐ brutus1213

I am a researcher on the AI/Systems side and I wanted to chime in. Transformers are amazing for language, and have broken all the SOTA is many areas (at the start of the year, some people may have wondered if CNNs are dead [they are not as I see it]). The issue with Transformer models is the insane amount of data they need. There is some amazing progress on using unsupervised methods to help, but that just saves you on data costs. You still need an insane about of GPU horsepower to train these things. I think this will be a bottleneck to progress. The average university researcher (unless from tier 1 school with large funding/donors) are going to pretty much get locked out. That basically leaves the 5-6 key corporate labs to take things forward on the transformer front.

RL, which I think this particular story is about, is an odd-duck. I have papers on this and I personally have mixed feelings. I am a very applications/solutions-oriented researcher and I am a bit skeptical about how pragmatic the state of the field is (e.g. reward function specification). The argument made by the OpenAI founder on RL not being amenable to taking advantage of large datasets is a pretty valid point.

Finally, you raise interesting points on running multiple complex DNNs. Have you tried hooking things to ROS and using that as a scaffolding (I'm not a robotics guy .. just dabble in that as a hobby so curious what the solutions are). Google has something called MediaPipe, which is intriguing but maybe not what you need. I've seen some NVIDIA frameworks but they basically do pub-sub in a sub-optimal way. Curious what your thoughts are on what makes existing solutions insufficient (I feel they are too!)

⬐ TaylorAlexander

Great comment thank you.

Yes unless the industry sees value in a step change in the scale on offer to regular devs, progress on massive nets will be slow.

Hooking things together is pretty much my job. I have used ROS extensively in the past but now I just hook things together using python.

But I consider what Tesla is doing to be pretty promising, and they are layering neural nets together where the output of three special purpose networks feed in to one big one etc. They call that a hydra net. No framework like ROS is required because each net was trained in situ with the other nets on the output of those nets, so I believe all compute logic is handled within the neural network processor (at some point they integrate standard logic too but a lot happens before that). Definitely watch some Karpathy talks on that.

And currently I am simply not skilled enough to compose multiple networks like that. So I could use multiple standalone networks, process them separately, and link them together using IPC of some kind, but it would be very slow compared to what's possible. That's why I say we're "not there yet". Something like Tesla's system available as an open source project would be a boon, but the method is still very labor intensive compared to a self-learning system. It does have the advantage of being modular and testable though.

I probably will hand compose a few networks (using IPC) eventually. I mean right now I am working on two networks - an RL trained trail following network trained in simulation on segmentation-like data (perhaps using Dreamer V2), and a semantic segmentation net that is trained on my hand labeled dataset with "trail/not-trail" segmentation. So far my segmentation net works okay. And a first step will actually be to hand-write an algorithm to go from segmentation data to steering. My simulation stuff is almost working. I built up a training environment using Godot video game engine and hacked the shared memory neural net training add on to accept image data, but when I run the sim in training on DreamerV2, something in the shared memory interface crashes and I have not resolved it. [1]

But all of this is a hobby and I have a huge work project [2] I am managing myself that is important to me, so the self driving off road stuff has been on pause. But I don't stress about it too much because the longer I wait, the better my options get on the neural network side. Currently my off road rover is getting some mechanical repairs, but I do want to bring it back up soon.

[1] https://github.com/lupoglaz/GodotAIGym/issues/15

[2] https://community.twistedfields.com/t/a-closer-look-at-acorn...

⬐ brutus1213

First off, amazing farm-bot project! I am looking forward to reading the details on your site.

Thx for the pointers on Tesla. Had not seen the Hydranet stuff. There was a Karpathy talk about 2 weeks back at a CVPR workshop .. he revealed the scale of Tesla's current generation deep learning cluster [1]. It is insane! Despite being in industrial research, I don't foresee ever being able to touch a cluster like that.

A lot of our current research involves end-to-end training (some complex stuff with transformers and other networks stitched together). There was a CVPR tutorial on autonomous driving [2], where they pretty much said autonomy 2.0 is all about end-to-end. I've spoken to a few people who actually do commercial autonomy, and they seemed more skeptical on whether end2end is the answer in the near-term.

One idea we toy with is to use existing frozen architectures (OpenAI releases some and so do other big players) and do a small bit of fine-tuning.

[1] https://www.youtube.com/watch?v=NSDTZQdo6H8 [2] https://www.self-driving-cars.org/

Sources: https://youtu.be/hx7BXih7zx8?t=639 and https://youtu.be/gZ2SeiLjaEc?t=941

This is anti-Tesla astroturfing, so I flagged it.

If you'd like a high-level technical overview of Tesla's FSD efforts, check out this talk by Andrej Karpathy from February 2020: https://www.youtube.com/watch?v=hx7BXih7zx8

There are literally thousands of videos of people filming the FSD beta doing amazing things. They are making great progress very, very quickly: https://www.youtube.com/watch?v=0ojlQi8zZxE

As you can see from videos like the one above, real-world data is needed to make FSD practical. There is a very long tail of things to get right, so blocking this progress with regulations would effectively outlaw FSD, or delay its development by a very long time. Given the number of road accidents that human drivers cause (think of just sleep deprivation and drunk driving), delaying FSD through regulation would ultimately cause a great number of uncessary deaths.

Removing the cameras may be a good idea, but note that their extensive fleet of camera-equipped vehicles is an important part of their potentially VERY lucrative self driving software development.

EDIT: I highly recommend watching head of AI at Tesla Andrej Karpathy's talk on their self driving architecture. "The Fleet" as a source of data is a vital functional part of their self driving software development. They may be able to calculate that including cameras and a computer on a car that will ship in high volumes would increase sales of their self driving option across all Tesla vehicles (including that low cost model) enough to justify the inclusion of those parts.

https://www.youtube.com/watch?v=hx7BXih7zx8

I'm sorry you missed out on one of the best investments of the decade, but there is a lot of room left for this to run.

Tesla's coming FSD and robotaxis still haven't been priced in. https://www.youtube.com/watch?v=hx7BXih7zx8

⬐ stopChanging

That's not two things, it's one, and I just don't see Tesla self driving panning out anytime soon. They've made wild claims about it for years now, but the actual product has progressed slowly. There have been far more failures than successes in that field over the past decade, and it's perpetually been given a "just a couple more years" timeline.

Better yet, let's get rid of cars altogether from city streets. This is coming along very nicely... https://www.theverge.com/2020/5/14/21257849/elon-musk-boring...

Autonomous driving is just around the corner, too, which means that there is no need to waste space on parking structures, either. Here's a Tesla engineer that gave a great technical talk demonstrating their progress as of early this year: https://www.youtube.com/watch?v=hx7BXih7zx8

EDIT: yes, there are clear reasons why having an three-dimensional underground structure will absolutely affect the social dynamics above-ground in a city, yet it remains true that it is possible to banish cars from downtown cores in the near future (i.e., say by 2030 or so), given the present shift to electrification and autonomous driving. These issues are worthy of debate, and I'm just trying to shift the conversation to what's current. I'm not trying to rain on anyone's parade! I love the goal of city streets for people, as exemplified by the beautiful laneways in Melbourne (11 minutes): https://vimeo.com/131396094

⬐ xkcd-sucks

If underground construction is so easy now, why not move all the people underground and leave the roads in place? It's much easier to build a lot of unconnected spaces vs a network, and being in a highrise is basically the same as being in an underground box

⬐ projektfu

Having lived in both a basement and a high rise, I disagree. The high rise was fabulous with tons of natural light, high above the noise of the city, with pleasant breezes and no risk of flooding. The basement was damp, had no natural light, transmitted all the noises from the street, and was stuffy because air wouldn’t naturally flow.

⬐ rootusrootus

I'm not going to live long enough to see autonomous cars in the hands of regular people. And I'm not particularly old yet.

Electric, I'm with you. We're at 2 out of 3 cars in our household being electric.

It's not the same: https://www.youtube.com/watch?v=hx7BXih7zx8

You can watch that talk and see the approach they're taking. Maybe you're more skeptical than they are about the near term possibility, but you can see that the work and progress is real.

The AGI risk is real too.

What Milton said doesn't make semantic sense, it's not a question of timelines.

https://intelligence.org/2017/10/13/fire-alarm/

⬐ gamblor956

I don't understand why it's relevant to watch a video of someone who is not Musk as a comparison to Milton.

I'm not claiming that the Head of AI Research at Tesla has said stupid things (I don't think he has), I'm saying that Musk has said stupid and nonsensical things that don't make semantic or scientific sense, and in comparison to Milton their hucksterism is just a difference of degrees.

⬐ fossuser

He's the head of AI and self-driving at Tesla because Elon hired him to do exactly that and understands the approach they're taking.

It's not a matter of degree, but one of kind.

⬐ tsimionescu

I firmly believe that some parts of the AI community have a vast overestimation of their capabilities.

AGI is not 50 years away - it is unknowably far in the future. It may be a century, it may be a millenium. We just know far too little about the human mind to make any firm prediction - we're like people in Ancient Greece or 1800s England trying to predict how long it would take to develop technology to reach the moon.

We are at the level where we can't understand how the nervous system of a worm works in terms of computation. Emulating a human-level mind is so far beyond our capabilities that it is absurd to even imagine we know how to do it, "we just need a little more time".

⬐ marvin

And yet most scientists would claim that there was no plausible, practical way to achieve a controlled fission reaction, with many flat-out stating it was impossible, at the same time that the first fusion reactor was ticking away in Chicago in 1942.

I agree it's not possible to have a good idea to the answer to this question, unless you happen to be involved with a development group that is the first to figure out the critical remaining insights, but it's more in the league of "don't know if it's 5 years or 50" rather than "100 years or 1000".

Predictions are easy when there are no consequences, but I wouldn't make a bet with serious consequences on critical developments for AGI not happening in the next decade. Low probability, probably, but based on history, not impossible.

⬐ tsimionescu

My reasoning is based on two things. For one, what we know about brains in general and the amount of time it has taken to learn these things (relatively little - nothing concrete about memory or computation). For the other, the obvious limitations in all publically shown AI models, despite their ever-increasing sizes, and the limitted nature of the problems they are trying to solve.

It seems to me extremely clear that we are attacking the problem from two directions - neuroscience to try to understand how the only example of general intelligence works, and machine learning to try to engineer our way from solving specific problems to creating a generalized problem solver. Both directions are producing some results, but slowly, and with no ability to collaborate for now (no one is taking inspiration from actual neural networks in ML, despite the naming; and there is no insight from ML that could be applicable in formulating hypotheses about living brains).

So I can't imagine how anyone really believes that we are close to AGI. The only way I can see that happen is if the problem turns out to be much, much simpler than we believe - if it turns out that you can actually find a simple mathematical model that works more or less as well as the entire human brain.

I wouldn't hold my breath for this, since evolution has had almost a billion years to arrive at complex brains, while basic computation has started from the first unicellular organisms (even organelles inside the cell and nucleus are implementing simple algorithms to digest and reproduce, and even unicellular organisms tend to have some amount of directed movement and environmental awareness).

This is all not to mention that we have no way right now of tackling the problem of teaching the vast amounts of human common sense knowledge that is likely baked into our genes to an AI, and it's hard to tell how much that will impact true AGI.

And even then, we shouldn't forget that there is no obvious way to go from approximately human-level AGI to the kinds of sci-fi super-super-human AGIs that some AI catastrophists imagine. There isn't even any fundamental reason to assume that it is even possible to be significantly more intelligent than a human, in a general sort of way (there is also no reason to assume that you can't be!).

⬐ fossuser

I think mixing in the human bit confuses the issue, you could have a goal oriented AGI that isn't human like that causes problems (paperclip maximizer).

Check out GPT-3’s performance on arithmetic tasks in the original paper (https://arxiv.org/abs/2005.14165)

Pages: 21-23, 63

Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that.

> evolution has had almost a billion years to arrive at complex brains

There are brains everywhere and evolution is extremely slow. Maybe the large computational cost of training models is similar to speeding that computation up?

> there is no obvious way to go from approximately human-level AGI to the kinds of sci-fi super-super-human AGIs that some AI catastrophists imagine.

It's worth reading more about the topic, it's less that we'll have some human comparable AI and then be stuck with it - more so that things will continue to scale. Stopping at human level might be a harder task (or even getting something that's human like at all).

> This is all not to mention that we have no way right now of tackling the problem of teaching the vast amounts of human common sense knowledge that is likely baked into our genes to an AI, and it's hard to tell how much that will impact true AGI.

This is a good point and basically the 'goal alignment problem' or 'friendly AI' problem. It's the main reason for the risk since you're more likely to get a powerful AGI without these 'common sense' human intuitions of things. I think your mistake is thinking the goal alignment is a prerequisite for AGI - the risk comes from the truth being that it's not. Also humans aren't entirely goal aligned either, but that's a different issue.

I understand the skepticism, I was skeptical too - but if you read more about it (not pop-sci, but the books from the people working on the stuff) it's more solid than you probably think and your positions on it won't hold up.

⬐ tsimionescu

GPT-3's performance on artihmetic is exactly one of the examples of how limited it is, and of how little the creators have tried to understand it. They don't even know if it has some (bad) model of arithmetic, or if it's essentially just guessing. I find it very hard to believe that it has an arithmetic model that works well for numbers up to the thousands but fails on larger numbers. More likely it has memorised some partial multiplication tables.

Getting back to the human bit, I'm using 'human' just as a kind of intelligence level, indicating the only intelligence we know about that can do much of anything in the world.

The paperclip maximizer idea still assumes that the AI has an extremely intricate understanding of the physical and human worlds - much better than any human's. My point was that there is no way at the moment to know if this is possible or not. The whole excersise believes that the AI, in addition to understanding the world so well that it can take over all of our technology, could additionally be so alien in its thinking that it may pursue a goal to this utmost extent. I find this combination of assumptions unconvincing.

Thankfully, the amount of knowledge we have about high level cognition means that I'm confident in saying that I don't know significantly less than, say, Andrew Ng about how to achieve it (though I probably know far less than him about almost any other subject).

I'm not claiming that AGI risk in some far future won't be a real problem. My claim is that it is as silly for us to worry about it as it would have been for Socrates to worry about the effects of 5G antennas.

⬐ fossuser

> More likely it has memorised some partial multiplication tables.

Did you read what I linked? (I don't intend this to be hostile, but the paper explicitly discusses this.) They control for memorization and the errors are off by one which suggest doing arithmetic poorly (which is pretty nuts for a model designed only to predict the next character).

(pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.”

> The paperclip maximizer idea still assumes that the AI has an extremely intricate understanding of the physical and human worlds - much better than any human's. My point was that there is no way at the moment to know if this is possible or not.

It seems less likely to me that biological intelligence (which is bounded by things like head size, energy constraints, and other selective pressures) would happen to be the theoretical max. The paperclip idea is that if you can figure out AGI and it has goals it can scale up in pursuit of those goals.

> I'm not claiming that AGI risk in some far future won't be a real problem. My claim is that it is as silly for us to worry about it as it would have been for Socrates to worry about the effects of 5G antennas.

I think this is a hard claim to make confidently. Maybe it's right, but maybe it's the people saying the heavier than air flight is impossible two years after the Wright brothers flew. I think it's really hard to be confident in this prediction either way, people are famously bad at this.

Would you have predicted gpt-3 kind of success ten years ago? I wouldn't have. Is gpt-3 what you'd expect to see in a world where AGI progress is failing? What would you expect to see?

I do agree that given the lack of clarity of what should be done it makes sense for a core group of people to keep working on it. If it ends up being 100yrs out or more we'll probably need whatever technology is developed in that time to help.

⬐ tsimionescu

> Did you read what I linked? (I don't intend this to be hostile, but the paper explicitly discusses this.) They control for memorization and the errors are off by one which suggest doing arithmetic poorly (which is pretty nuts for a model designed only to predict the next character).

I read about this before. I must confess that I had incorrectly remembered that they had only checked a few of their computations for their presence in the corpus, not all of them. Still, they only check for two possible representations, so there is still a possibility that it picked up other examples (e.g. "adding 10 with 11 results in 21" would not be caught - though it's still somewhat impressive if it recognizes it as 10 + 11 = 21).

> It seems less likely to me that biological intelligence (which is bounded by things like head size, energy constraints, and other selective pressures) would happen to be the theoretical max. The paperclip idea is that if you can figure out AGI and it has goals it can scale up in pursuit of those goals.

Well, intelligence doesn't seem to be so clearly correlated with some of those things - for example, crows seem to have significantly more advanced capabilities than elephants, whales or lions (tool use, human face memorization). Regardless, I agree that it is unlikely that humans are a theoretical maximum. However, I also believe that the distribution of animal intelligence to brain size may suggest that intelligence is not simply dependent on the amount of computing power available, but on other properties of the computing system. So perhaps "scaling up" is not going to be a massive growth in the amount of intelligence - that you need entirely different architectures for that.

> Would you have predicted gpt-3 kind of success ten years ago? I wouldn't have. Is gpt-3 what you'd expect to see in a world where AGI progress is failing? What would you expect to see?

I don't think GPT-3 is particularly impressive. I can't claim that I would have predicted it specifically, but the idea that we could ape human writing significantly better wouldn't have seemed that alien to me I think. GPT-3 is still extremely limited in what it can actually "say", I'm even curious if it will find any real uses that we don't already outsource as brain-dead jobs (such as writing fluff pieces).

And yes, I do agree that this is a problem worth pursuing, don't get me wrong. I don't think lots of AI research is going in the right way necessarily, but some is, and some neuroscience is also making advances in this area.

You set goals and work towards them.

Becoming an inter-planetary species doesn't happen in a year.

I want SpaceX to succeed and they have a track record of execution such that I now believe they really can. I was hopeful before (and if you listen to Musk talk about it he didn't think they'd be able to really pull it off early on either but figured they'd at least make progress towards it even if they failed), but now I think a mars colony is a real possible outcome.

It's not a bold take to just state something is impossible until it happens, that's pretty much the default.

The bold take is to look at what might be possible and execute goals in pursuit of that.

For SpaceX this means reusable rocket technology to bring costs down (massive success here has them ahead of everyone else). Starlink as a revenue source is also a really good approach.

For Tesla it's the 'master plan' of roadster -> model s -> model 3, reinvesting in infrastructure and battery technology with vertical integration to build out superchargers and drive costs down. This has been massively successful and their EVs (particularly the model3/y) have no equal at any price point EV or gas. The level 5 autonomy was really a bonus on top of that EV transition that they've added to, and if anyone can pull it off it will be Andrej Karpathy and the fleet of Tesla's they can train with (https://www.youtube.com/watch?v=hx7BXih7zx8).

Bullshit and really big ideas can sound similar, but that doesn't mean they are - there's a lot of value in being able to tell the difference.

In the case of Nikola, they're just lying to enrich themselves and taking advantage of those that can't see the difference between a person like Trevor Milton and a person like Elon Musk.

⬐ flunhat

I agree with you that they’re working towards these goals. And there’s obviously a ton of daylight between Musk and Milton.

But the point I’m making is more specific than that: you can’t call them successes because they haven’t achieved their goals yet. (But they haven’t failed yet, either. The jury is still out.)

⬐ fossuser

This is just disputing definitions: https://www.lesswrong.com/posts/7X2j8HAkWdmMoS8PE/disputing-...

As far as daylight between Musk and Milton, it's not a matter of degree, but one of kind.

If you actually believe that Tesla will fail to deliver full self-driving in the coming years, I have two questions.

1. have you watched this entire technical presentation made by Andrej Karpathy, Senior Director of AI at Tesla? https://www.youtube.com/watch?v=hx7BXih7zx8

2. if you understand what you've seen in that video, why do you think Tesla will fail?

⬐ kirillzubovsky

As a Tesla owner, and regardless of the presentation, I have serious doubt that Tesla will be able to solve all the edge cases. Machine learning needs data, and with my family in the car, I don't want it to make a decision whether or not to break, while heading in ongoing traffic. I want the car to know.

Right now, 99% reasonable self-driving doesn't bother me because I am always in control, and I already know where the car is going to mess up and get ready to take control ahead of time. It works, and it works really well.

But the different between 100% and the 99% is all the difference that matters, and it's colossal.

I hope they can figure this out, but I don't know how. I hope they do.

⬐ fluffything

1. Yes

2. Some company will succeed at L5 FSD, eventually. Just because it can be done does not mean that Tesla will do it. Tesla has been selling Vaporware for a decade, and they are investing a lot of resources in trying to get FSD to work with the hardware they sold customers 5 years ago to save face, while right now we don't know, and many do not believe, that such hardware suffices. Companies that have not tied themselves to such promises are much more flexible from a technology point-of-view to make quicker progress.

For all we know, next year somebody could ship a smaller, better, and cheaper lidar systems, that might give every manufacturer willing to use them a huge advantage over Tesla. Tesla would be in a very tight spot to incorporate such technology into their existing customer base.

I don't think we will see L5 in the next 10-20 years and a lot of things can happen in such a time-frame technology wise.

⬐ detaro

What's "the coming years"? 5 years after they first made the claim? 5 years from now? 5 years from 5 years from now?

⬐ maxharris

2024. Cathie Wood's bull case for everything going right for Tesla is 24k/share in 2024. She also breaks out everything else, shows what happens to the share price if autonomy doesn't happen, or if they don't keep building gigafactories, etc.

⬐ detaro

Everything going right for Tesla would have been Elon's 2015 prediction of "in two years" coming true in 2017/18 ;)

Do you have a link that explains why the predictions now will be better? I haven't seen much to suggest it, but also honestly kinda stopped paying attention.

⬐ Dahoon

For one because they said it would be here already but they keep moving the goal post.

⬐ maxharris

Yeah, but that's how every single thing that Elon Musk does is.

I followed all those failures at SpaceX before they landed the first booster, or the Model 3 intro, or fairing catches, or the original Model S. These things all took a lot longer than you'd expect from his comments, but they all happened!

⬐ ryan93

Google has a much larger and better funded team that still doesnt seem close. Karpathy is no doubt smart but google has like 10 karpathys for every one TESLA has.

⬐ bflesch

The problem is these 10 karpathys have no vision, otherwise they would've joined Tesla.

⬐ maxharris

Did you watch the video? Waymo is stuck using lidar, and the video explains why that's a dead-end.

(Want to keep in touch about this bet? I'm maxharris9 on twitter.)

⬐ catalogia

I skimmed the video. It's doing what I expected, knocking down a goofy strawman of LIDAR-only while ignoring the obvious camera/LIDAR sensor fusion. The depth map Tesla is getting from stereoscopic vision is pretty shoddy; sensor fusion with LIDAR is the obvious solution. The reason Telsa resists this is because they want to market their cars as having all the requisite hardware and acknowledging the usefulness of LIDAR wouldn't let them market their cars that way profitably.

⬐ maxharris

Hmm, looks like Tesla actually does do sensor fusion, just not with lidar: https://news.ycombinator.com/item?id=19803817

I also think that being so cynical about Tesla's motives is pretty short-sighted from an investment perspective. In the long-term, they don't win if they don't get this right.

⬐ catalogia

Their radar/ultrasound has awful angular resolution. That's where LIDAR excels.

This is why Telsa cars run into trucks parked across the street. Their stereoscopic depth map is shoddy and the radar or ultrasound has awful angular resolution that can't tell the difference between an object parked next to the street and one parked in the middle of the street.

> "In the long-term, they don't win if they don't get this right."

They've been claiming they're on the cusp of getting it right in the short-term for years. So far, my cynicism has served me well.

⬐ maxharris

I have 110 shares, and I'd love to talk to you about how that's going in 2024. I'm maxharris9 on twitter

⬐ agakshat

Whether or not a camera-based solution will work reliably in the future is not the issue at hand here.

The issue is the vast, vast gap between the current capability of Autopilot vs. what it is marketed as, and how this false advertising can literally kill the average consumer.

⬐ grecy

> Whether or not a camera-based solution will work reliably in the future is not the issue at hand here.

Actually, it is precisely the issue at hand:

From the ruling:

> The Munich court agreed with the industry body’s assessment and banned Tesla Germany from including “full potential for autonomous driving”.... in its German advertising materials.

So Tesla are not allowed to advertise that it has the potential to do it in the future.

⬐ PunchTornado

sort of, but it is the gap that is the issue. Today, tesla seems so far away from that goal that this amounts to false advertising. it's so far that one can actually doubt that they will achieve it in our lifetime.

whether or not you think tesla is close, just drive a tesla on a narrow road in Germany.

https://youtu.be/hx7BXih7zx8?t=513

⬐ kordlessagain

> complicated when you get to the long tail of it

Well, there's the problem right there.

⬐ lokedhs

Watching this presentation did not make me any more confident about going in a self-driving car.

Based on the way it was presented, I got the feeling that they are just essentially manually identifying cases and addressing them as they see them. Is that solution really helping to make the system more robust when encountering an unexpected situation?

LIDAR vs camera is a red herring. The fact that Elon and his fan club fixate on this shows you how little they understand about self driving. The fundamental problem is that there is no technology that can provide the level of reasoning that is necessary for self driving.

Andrej Karpathy's most recent presentation showed how his team trained a custom detector for stop signs with an "Except right turn" text underneath them [0]. How are they going to scale that to a system that understands any text sign in any human language? The answer is that they're not even trying, which tells you that Tesla is not building a self-driving system.

[0] https://youtu.be/hx7BXih7zx8?t=753

⬐ reanimated

They have hired most of the industry talents, so I think it's quite silly to state about how little they understand about this. In my opinion nobody except Tesla and Waymo has more knowledge of this field.

⬐ anchpop

Why does it need to work in any human language? It isn't as if self driving cars need to work on Zulu road signs before they can be rolled out in California. I'd be surprised if they ever needed to train it on more than 4 languages per country they wanted to roll out to.

⬐ aeternum

A surprising number of human drivers would also not be able to 'detect' that 'except right turn' sign. Only 3 states offer driver's license exams in only English and California for example offers the exam in 32 different languages.

Even so, it is quite possible to train for this in general. Some human drivers will notice the sign and will override autopilot when it attempts to stop, this triggers a training data upload to Tesla. Even if the neural net does not 'understand' the words on the sign, it will learn that a stop is not necessary when that sign is present in conjunction with a stop sign.

Here's the most recent talk by Karpathy about this topic: https://www.youtube.com/watch?v=hx7BXih7zx8 (there are more if you google for "karpathy talk")

Here's the summary via 2 examples.

1. Example of gathering data that needs further labeling

To implement neural network (NN) to recognize stop signs they program the cars to recognize things that look like stop signs and deploy that to the fleet of over 600 thousand cars on the road. The cars send those "might or might not be a stop sign" images back to tesla and they get manually labelled and added to a training set. They gather enough images and the "recognize stop signs" feature is done. Apply the same logic to other recognition tasks: recognize cars, pedestrians, animals, speed signs, traffic lights etc.

Yes, labeling is expensive but everyone else (including Waymo) has the same cost.

But Waymo has even bigger cost of driving around to collect those images.

Tesla makes gross profit on every car they sell and they got 600 thousand people driving for them for free. Waymo has 600 cars, each of them reportedly costing over $200k and they have to pay drivers at least minimum wage for each hour of driving.

That's why Google is reportedly is spending $1 billion a year on Waymo and why they need $3 billion of additional investment to keep going.

2. Example of gathering data that doesn't need labeling

Consider implementing NN to recognize cut in i.e. other cars entering your lane in front of you.

They deploy a first version trained on small sample, running in shadow mode i.e. it makes predictions for cut ins but doesn't act on it.

When it makes wrong prediction, it sends the clip of the cut in back to Tesla. This doesn't need manual labeling. They know whether the car did cut in or not so they can rewind time and automatically label the past car action.

⬐ gizmondo

Thank you.

It's not luck.

I have a degree in molecular biology, and I took a lot of elective courses in immunology, oncology, virology. I worked in a research lab on HIV vaccine for two years after undergrad before I switched back to software engineering. So I was able to keep a cooler head about the virus than most, which enabled me to buy intelligently. I worked damn hard to earn that degree and in my time in the lab. I was a twitchy premed student that needed every grade to be perfect. NONE of the effort I poured into those six years of my life was luck. It was pure sweat.

In the immediate future, I expect that the dip in advertising will cause Q1 earnings from Google and Facebook to be lower than expected. As the Fed continues to inflate stocks, this will cause a lot more buying of TSLA and AAPL.

In the 2020s, Tesla will continue to soar because:

1) they have a huge lead in battery tech. they're about to announce that they've achieved the $100/kWh mark, which will let them sell electric cars that go as far as gas ones but don't cost more.

2) batteries are crucial for amazing tech like the emerging eVTOL scene. it's possible Tesla may enter that space and dominate it, because batteries determine everything there, too. in the meantime, I bought some shares of eHang (Chinese company, listed on the Nasdaq under EH)

3) they have a huge lead in autonomous driving. LiDAR is a dead-end, and they're ahead of everyone else there, too: https://www.youtube.com/watch?v=hx7BXih7zx8

4) demand is not a problem at all, especially in China (where the latest new factory is). demand for iPhones was just fine in 2008-2009! remember Ballmer reacting with, "$600 for a phone!?" when he was asked about the iPhone in 2007?

5) they now sell car insurance. they're using all the data collected by their cars to offer significantly lower rates to customers

6) they're going to get into home HVAC, which will integrate beautifully with the Powerwall and solar roof products. I have been looking at buying a cheap off-grid lot up in the mountains to build a home on (when the price is right!), and I absolutely will buy that and avoid paying significant costs to get the lot hooked up to the grid.

7) their competitors will continue to fail to invest in EVs, even though their futures depend on it. this is because they are loaded up with tons of debt, while Tesla has a lot less

8) the site for Gigafactory Berlin has been fully cleared, and it will be under construction very soon. there's another factory coming in the US later this year.

9) I should at least mention the new products! Model Y is a definite hit, and it's clear that the Cybertruck will be! the new semi was delayed only because they simply can't produce enough batteries for it and the Model Y at the moment. that's gonna change!

10) they have a new analyst covering them at Goldman, and he's bullish. I expect TSLA to join the S&P 500 this year, which will trigger a massive wave of institutional buying.

I think ARK Invest has it right. $24k/share by 2024. Out of the scenarios they give, I think their middle-range estimate of $3.4k/share by 2024 is my worst-case.

Tesla is $54.74% of my portfolio right now, and if I could go back to March and do it all over again, I'd make it 100%.

If you want to keep in touch and lord it over me when I go bankrupt, I'm maxharris9 on Twitter.

⬐ ipi

For me operation vacation is the interesting point of the presentation apart from the regular ML stuff. That's something which we need to integrated in our regular development workflow.

⬐ shermanmccoy

Why didn't he answer the first question in the Q&A session?

⬐ strangecosmos

I thought he did?