They hired some uninspired losers to lead their self-driving efforts. They basically were weak, looked into it and figured it's impossible.
There's nothing wrong with knowing your own limitations and acting accordingly. Had Mr. Zuckerberg had that sort of self-knowledge, he might have avoided lighting 80-billion-with-a-b dollars on fire and renaming his social media company after a technology that nobody wanted.
And human drivers are NOT, "on average, really bad".
That is true, but it's a little more complicated than that.
The average driver is pretty good, 99.9% of the time. The other 0.1% of the time, they are tired, or angry, or momentarily distracted, or just looking the other direction from where they need to be looking under those particular circumstances. It's those occasional blips of inattention that cause lead to most accidents occurring.
But I value even a 1 line change that clears a static analysis warning if even in practice it was impossible to trigger the issue in a real system.
Yes, I agree. The problem with thinking "this bug seems harmless, because I can't imagine how anyone could exploit it" is in the "I can't imagine" part; my imagination is limited to what is covered by my mental model of how computers work, but an attackers' ingenuity is not.
In particular, the C/C++ optimizer is a devious beast, and will exploit any opportunity to make the code more efficient, even if that means doing things that are wildly unintuitive to a naive human reader -- and it sees any instance of undefined behavior as an opportunity to exploit.
(C) is the only one of those the AI companies are really concerned about -- they want to stave off model collapse for as long as possible. The problem is that AI generates so much content that the ratio of AI-generated data to human-generated data keeps rising, and sooner or later there simply won't be much human-generated data left to anything to consume. At that point they'll either have to figure out to ingest AI-generated content without causing degradation, or they'll have to accept that AIs have gotten as smart as they will ever get, and it's all downhill from there.
I always figured that people who genuinely believed their employers were likely to kill literally everybody in the near future might burn down a bit more than that, but so far it's mostly been tweets
Let's be more specific about how and why this could happen? What scenarios could lead to the total destruction of mankind?
Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.
One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.
It's really the same problem that we see with quantum physics -- the subject matter is esoteric enough that common sense can't be used to determine whether what you're listening to is really advanced theory or utter BS (or some combination of the two); they both sound the same, and the situation isn't helped by the presence of large numbers of people who think they know what they are talking about but don't.
I think the distinction they are making is between technology that kills people, and technology that entertains people. If someone fitted actual weapons to a Doom engine, they'd probably object.
This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.
But here's the part I find interesting: why assume a superintelligent AI would actually want to kill us?
Let's make the hopeful assumption that the superintelligent AIs will not "go rogue" in any way, but will correctly try to carry out their tasks as assigned by the people who designed and trained them.
Pretty much every major nation has military AIs, at this point. So the Chinese AIs are working hard on how to defeat the Americans, and the American AIs are working hard on how to defeat the Russians, and so on. In the final analysis, no matter where you live, there is probably an AI that has been explicitly tasked by its operators with figuring out the most effective ways to do you in. Not because they've "gone wrong", but because they're working as intended.
The size of the universe is increasing...
but the rate at which it is increasing is decreasing...
but the rate at which the rate is decreasing is increasing...
but the rate at which the rate of the rate is increasing is decreasing...
but the rate at which the rate of the rate of the rate is decreasing in increasing...
[... and so on, ad infinitum...]
That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.
I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.
And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.
The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.
They don't care because they don't believe that.
When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.
> approximately eight hundred billion
God help those who do not help themselves. -- Wilson Mizner
