Comment Re:Directionally Right, Specifically Wrong. (Score 1) 160
It wasn't very good at counting the costs to anyone, and was pretty much ignorant of any costs to anyone else.
I don't believe today's models are AGI, but "not understanding cost" is a dangerous assumption to make. If you read the METR analysis of the Hugging Face attack, the agents used were very much aware of their own compute budgets and worked to maximize what the swarm had available to achieve the task.
And the training they were given and the prompts provided by humans provided them with the internal rewards.
Just because the humans working on this don't know how the model stored all that data. That was almost certainly never fully tested, does not make them not responsible for the outcome of using a tool they do not fully understand when they prompt the task to be completed.
AI will not kill us. The humans making and using it will.
They clearly do not know what they are doing and should be prosecuted.