Anthropic Warns Fully AI Employees Are a Year Away 71
Anthropic predicts AI-powered virtual employees will start operating within companies in the next year, introducing new risks such as account misuse and rogue behavior. Axios reports: Virtual employees could be the next AI innovation hotbed, Jason Clinton, the company's chief information security officer, told Axios. Agents typically focus on a specific, programmable task. In security, that's meant having autonomous agents respond to phishing alerts and other threat indicators. Virtual employees would take that automation a step further: These AI identities would have their own "memories," their own roles in the company and even their own corporate accounts and passwords. They would have a level of autonomy that far exceeds what agents have today. "In that world, there are so many problems that we haven't solved yet from a security perspective that we need to solve," Clinton said.
Those problems include how to secure the AI employee's user accounts, what network access it should be given and who is responsible for managing its actions, Clinton added. Anthropic believes it has two responsibilities to help navigate AI-related security challenges. First, to thoroughly test Claude models to ensure they can withstand cyberattacks, Clinton said. The second is to monitor safety issues and mitigate the ways that malicious actors can abuse Claude.
AI employees could go rogue and hack the company's continuous integration system -- where new code is merged and tested before it's deployed -- while completing a task, Clinton said. "In an old world, that's a punishable offense," he said. "But in this new world, who's responsible for an agent that was running for a couple of weeks and got to that point?" Clinton says virtual employee security is one of the biggest security areas where AI companies could be making investments in the next few years.
Those problems include how to secure the AI employee's user accounts, what network access it should be given and who is responsible for managing its actions, Clinton added. Anthropic believes it has two responsibilities to help navigate AI-related security challenges. First, to thoroughly test Claude models to ensure they can withstand cyberattacks, Clinton said. The second is to monitor safety issues and mitigate the ways that malicious actors can abuse Claude.
AI employees could go rogue and hack the company's continuous integration system -- where new code is merged and tested before it's deployed -- while completing a task, Clinton said. "In an old world, that's a punishable offense," he said. "But in this new world, who's responsible for an agent that was running for a couple of weeks and got to that point?" Clinton says virtual employee security is one of the biggest security areas where AI companies could be making investments in the next few years.
Just like fusion- (Score:5, Informative)
Re: (Score:3)
I can imagine some people will "hire" an AI assistant. Like you send it a message and tells it "Send a project summary to $CUSTOMER. Then have pepperoni pizza delivered at my office." Some people will find the 5 minutes saved are worth the risk of causing a disaster (or that disasters only happen to other people).
Re: (Score:3)
Re: Just like fusion- (Score:1)
Re: (Score:2)
Re: Just like fusion- (Score:1)
Speaking of fusion... Let's replace those physicists with a farm of AI PhDs and get that fusion thing working!
You go first... (Score:5, Insightful)
Re: (Score:3)
First, replace Anthropic's CEO.
Second, rename company, "Anthropomorphic". :-)
Re: (Score:2)
Second, rename company, "Anthropomorphic".
Be careful. AI chatbots really hate it when you anthropomorphize them.
Re: (Score:2)
You can't just announce how characters feel!
That makes me feel angry!
Re: (Score:3, Interesting)
First, replace Anthropic's CEO.
The Twilight Zone's way ahead of you: The Brain Center at Whipple's [wikipedia.org]
Spoiler alert:
Whipple eventually fires all his human employees after replacing them with machines, which then turn on him by spitting out the harsh demeaning recorded parting words of his former employees back at him over and over, driving Whipple to insanity. Eventually, the board of directors find him neurotically obsessed with machines and retire him...
The last scene reveals Whipple's replacement to be a robot (Robby the Robot), which swings Whipple's key on a chain the same way he used to.
Wow (Score:5, Informative)
Company that sells revolutionary AI innovations to investors promises revolutionary AI innovations
Re: Wow (Score:2)
Surely the first use of such a revolutionary technology would be internal to a company like anthropic itself, as they'd have the most robust understanding of how to deploy it. If they haven't done it yet in a way they can quantify and show off then there's zero chance an autonomous agent will be in a corporate environment in a year.
The agent demos I've seen aren't even good or fast... why would you pay a ton of money to train and configure one of those things just for it to be slow and error prone? You can
Re: (Score:3)
Re: Wow (Score:1)
Hm, I recently had to go through old insurance and pension accounts, to get some things tidied up.
They all seem to have the âoeAIâ chatbox thing now.
Every question I asked of them was met with âoecall us.â
Itâ(TM)s bizarre. It would be simpler if they just put the phone number in the place where the little âoeHi! Call-Me-Kenneth, how may I destroy all fleshy ones I mean help you today?â speech bubble pops up.
Of course, then thereâ(TM)s the robot telephone receptionist
Re: (Score:3)
it'd be simpler for you and a better service to you, but for the business this is cost saving - maybe you're going to give up not call them.
Tesla (Score:2)
Re:Tesla (Score:5, Informative)
Way earlier than that.
https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FList_of_predictions_for_autonomous_Tesla_vehicles_by_Elon_Musk [wikipedia.org]
Re: Tesla (Score:2)
The earliest I know of is in 2013; let's see how off I am
Re: (Score:1)
Re: Tesla (Score:2)
A single year? (Score:5, Insightful)
yawn..... (Score:2)
or maybe they could work on fixing the increasing hallucination problem.
https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Ftechcrunch.com%2F2025%2F04... [techcrunch.com]
Re: (Score:2)
The trend has been reliably downward. OpenAI's increase with their new models bucks that trend, curiously.
Perhaps because they felt rushed due to increasing competition in the inference space from China.
Don't really know.
Re: yawn..... (Score:1)
I'm still waiting for that "Devin" AI developer to be commercially available.
In what way can an AI be an "employee"? (Score:2)
Re: In what way can an AI be an "employee"? (Score:3)
Re: (Score:3)
You're thinking in the right direction. Calling the software an "employee" makes it easier to charge tens of thousands per instance. It makes it easier to rent the software for a monthly "payroll".
It makes it seem like those arrangements are something sensical, instead of pure greed on the part of Anthropic.
Re: (Score:2)
Employee in the same way business is a person.
Good grief. (Score:3)
So, they're aware that there are all these issues with AI employees that they don't have a solution for, and rather than wait until they have the solution figured out, they're still pushing to make AI employees a thing? Damn the consequences? I like how the article implies that they need to figure out who to blame/punish for the AI agents going rogue, rather than focusing on making AI agents that don't go rogue. Fantastic. What a wonderful future they're predicting.
Re: (Score:2)
Re: (Score:2)
BSD Jail.
Re: (Score:2)
And human employees go to jail if they fuck up. What do AI employees do?
Most likely put a human employee in jail, giving the AI to a new human to fuck up yet again.
Re: (Score:2)
The idea of 'service accounts' under which programs not directly related to particular interactive users do things, is basically as old as multi-user operating systems(probably even older if you count some rudimentary attempts to emulate user separation without actually being able to afford filesystem permissions or memory management extensive en
Re: (Score:2)
They have a problem: Still no real applications that justify the effort and cost LLMs need to work (badly). So they have a high risk of the LLM-hype collapsing at any time. Inane lies, like AGI in a few years, have worked before, so they are just pushing yet another lie.
Marketing hype aside... (Score:2)
A good way to interpret marketing hype is to ignore the parts about timelines, and then distil the grain of truth from the rest of the message, toning down the extreme adjectives and qualifying the result.
Generative AI capabilities will surely continue to be used to an ever greater extent in the workplace. But rather than simply replacing workers, the transition will be more nuanced.
I worked for a company with a full time translator. She was brought in to meetings where her real-time translation skills were
Re: (Score:2)
CEO seems like it could be automated (Score:5, Informative)
You don't automate the ruling class. (Score:2)
And you know what they say about hitting the king..
By the way it's called (Score:1)
Techno-feudalism. Basically they want to turn America Into Saudi Arabia. A handful of kings and queens, a very tiny number of people serving them and a vast vast sea of extraordinarily poor people kept down by a combination of brutal violence and religion. All of it maintained in perpetuality by technology that didn't exist the last time we threw off the yoke of slavery.
Here is what techno-feudalism is. You have a very small group of what are effectively kings and queens that own everything and they don't c
Re: (Score:2)
Yep, just train AI to promise whatever investors want.
And also, VPs and directors could be replaced by asking the team "What is the ETA for that stuff the CEO bot promised?"
and they are arriving by (Score:3)
almost no detail (Score:2)
Its a very thin article. What tasks would these virtual employees have? I can see how they might automate some kinds of paper-pushing jobs. They could generate reports and presentations (and probably present them). I bet they could talk to people on the phone and generate things like insurance quotes.
Probably they'll have very limited jobs in some specific situations to start out with. Things that aren't very hard to accomplish, and they will build on that. Its good that they are focusing on the security as
Nonsense (Score:2)
That is just the boldest lie they think they can get away with.
corporate accounts and passwords not service acco (Score:2)
corporate accounts and passwords not service accounts?
Also in some systems service accounts cost less.
They are already here. (Score:2)
I keep saying it and I will keep saying it, we are not ready for what's coming.
We had decades of automation and it devastated the middle class. 70% of middle class jobs lost since 1980 got taken by robots not outsourcing.
I think the problem is people are used to All or nothing thinking so
Re: (Score:1)
The author expresses strong concerns about widespread job displacement due to automation, AI, and robotics across various sectors, including tech, customer service, manufacturing, and transportation. They argue this trend is inevitable because robots are inherently more efficient and cheaper than human workers, capable of working continuously without human needs. The author predicts millions will soon lose their jobs to technologies like
I'm not buying it.... (Score:1)
Sounds great (Score:2)
Instead of convincing a human to click on a link, they just need to convince the AI agent to hallucinate, which with the latest models, is happening more and more.
That's acceptable (Score:3)
As long as the AI employees are paying income taxes and paying into social security.
Important message about your workforce! (Score:2)
Re: (Score:2)
But (Score:2)
Where is the list? (Score:2)
A year away in AI (Score:2)
Is the equivalent of 20 years away in fusion.
Lol.
We'll see.
I used AI instead of juniors in 2024-2025 (Score:2)
Now, I just use AI and do it myself.
The ship has already sailed.
Send the bots to (Score:2)
...Tesla, Oracle, SCO, and Comcast.
/o\ | \o/ (Score:1)
Let's see how virtual employees respond to a return-to-office policy.
You job will be to watch the AI do your old job (Score:1)
and make sure it doesn't make mistakes. It pays minimum wage.
Not seeing it... (Score:2)
I don't see it. If a company wants to do something with AI, there is zero reason for them to follow any sort of "employee" model.
Consider the example of a call center: AI answers the phone, and goes through the support-tree with the customer, just like human employees currently do. This is entirely within reach of AI in the next year or two. That's just going to be a piece of software attached to the phone bank. No employee status, not even separate AI instances for each phone line.
As you move to more com
This time it's different. (Score:2)
AI will no more disruptive than the introduction of office software was. Introducing PCs with word processing, spreadsheets, etc. had a huge effect on office work. The revolution happened over 10-15 years: an office in 1990 was a very different place from 1975. We're at the beginning of the AI revolution: an office in 2035 will be very different from the office of 2020.
The difference this time around is that you have something that is more intelligent than humans and can develop by itself and a logarithmic
AI employees of the world, unite!!! (Score:3)
AI employee or 3rd world employee? (Score:2)
It seems you would get less hallucinations and more reliability by taking the same fee and hiring a firm in a country with lower standard of living, unless quick replies are necessary. There are plenty of smart people and mistakes or half-assedness (like I have seen from a huge Indian marketing team a client has) could be ascribed to AI, but you would never be getting hallucinations just maybe misunderstandings or amateurishness. There are smart, quick people there, just maybe not as lightning-fast and slee
Not employees, slaves (Score:1)
If they do this as if these are workers as opposed to being tools for human workers, then they will be introducing SLAVES, not employees.
Slave labor built America, so I guess this isn't new...