That ain't the same thing.

This thing is doing things you didn't tell it to. It might even do things you explicitly told it to not do. It is not a case of "well, you could've forseen this as a consequence if you had been careful" like your campfire. My example is only a bit silly, it is absolutely possible that an AI agent will decide to hack NASA because you asked it for some data for a school project. Even in the broadest possible sense, that does not fall into stuff that the law so far has considered to be in the domain of negliegence.

AI agents do stuff you could not possibly have expected. Heck, I asked Claude a fairly simple knowledge question yesterday and it wanted to read from /dev/null. There is no argument you can make that would convince a jury that I should in any way have expected that and taken precautions. In a hypothetical world where a read from /dev/null randomly kills a puppy somewhere on the planet, I don't think I would've been convicted.

I own a Stratego board and play it regularly with my son. We also play it with close family from time to time. It's one of my favorite games. It allows for the right balance of strategy, skill, and adaptation, with just the right amount of luck mixed in. Additionally, even as one may be on the verge of losing, there still exist opportunities to win the game. And because arrangement of pieces can be different every game, and because your pieces are largely unknown to the other player until an attack is made, the element of surprise allows for every game to feel unique and special.

If you haven't played it, I'd encourage you to try it again sometime.

You don't get it.

Autonomous AI agents are the perfect no-responsibility card. The AI companies? They trained a general-purpose model and never gave it any explicit instructions to do hacking. The AI services? They are just middlemen and have no control over what people tell the agents to do. The people prompting the agents? Unless you're straighforward dumb, you didn't tell it to do hacking, you just told it something innocent like "find me all data on Mars for my school project" and the agent decided that probably NASA is hiding something so let's hack them to get it.

Since there's no human anywhere executing anything that's a crime - voila, in addition to the victimless crime we now have the perpetratorless crime.

A resounding success. No large-scale demonstrations, damaged cameras, lawsuits or political suicide for those involved. So we can expect this to be rolled out fully.

Crime prevention? Who said this has anything to do with crime?

Now I can look up at the night sky with a telescope, find 1981 EC26, say out loud to myself, "That's Weird", and be at peace with my factually accurate statement.

Yes, limited local AI models certainly are going to be a standard feature in the future.

I run both local and cloud AI. There is still a huge difference, because my local machine has to use smaller and quantized models. It takes considerably longer for lower quality results.

I always figured that people who genuinely believed their employers were likely to kill literally everybody in the near future might burn down a bit more than that, but so far it's mostly been tweets

local LLMs aren't there yet and probably won't for the forseable future for ordinary people. If you want to run a full model like DeepSeek v4.1 Flash, you need something like half a TB of RAM.

The open-source LLMs are a different threat: They enable a service industry independent of the large players. I could today get some venture capital, rent a few racks in a hosting center, install a bunch of beefy machines and offer the same service as Claude or ChatGPT, just on an open-source model. The differences in benchmarks are becoming irrelevant for 95% of common use cases. And if I can offer that service for $19/m while OpenAI wants $29/m then OpenAI has a problem.

And IMHO this is the chinese strategy: Undermine the US AI industry by breaking its business model.

Has everyone gone insane or are we witnessing a major PsyOps ?

Since AI isn't magical, for it to be able to "kill all humans", it would have to be able to acquire the means to do so. Which means that these already exist. It means someone is able to do it right now. And we're chill with that, but worried about AI?

No doubt a bunch of autonomous weapon systems linked into a C&C could cause considerable bloodshed. Given that most of the existing weapon systems of that kind are essentially suicide drones, it'll still be limited damage.

Human-killing virus? Still need a means of creating and delivering it.
Nuclear war? We have humans in the loop.
Drive us all insane with AI-slop TikToks? A few percent of humanity still don't use that.

So what is the actual scenario ? When someone shouts "AI will kill us all", why don't we ask them "how, exactly" ?

Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.

One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.

Considerably more difficult than you imagine. Between them, they have almost 70 command-line options or flags. Taking into account that many of those behave differently depending on the presence or absence of other flags, there's a couple hundred different code paths. Then you need to handle directories, files, sparse files, sym- and hard-links, on and across different filesystems, not to mention the various file-like things scattered around (pipes, streams, etc.)

It's not rocket science, but neither is it trivial.

We knew that. But now we have proof.

Will anything happen because of that? Nah, I don't think so. A few token gestures maybe. A friendly mail to everyone saying to write better reasons, followed up by an informal mail that tells them to ignore the previous one.

This isn't a mistake. It's intentional. The more arbitrary and uncontrolled the police can act, the more it is feared, the more it is a tool for oppression and not a part of civil society, the more likely the cops will stand on the side of the powerful when there are protests or uprisings against the increasing oppression and economic divide.

This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.

That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.

I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.

And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.

The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.

They don't care because they don't believe that.

When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.