This is just insanity marketing. "AI" doesn't need "safety". That's ridiculous. It's the same argument for censorship that gets trotted out all the time "We can't have kids reading the Anarchist's Cookbook!" In this case it's even more disingenuous: it's a combination marketing ploy (our AI is so good it's scary!) and demand for regulatory capture (Our competitors AI will be even scarier without government stepping in and giving us the only keys!). The fact is that "AI" by which I mean LLMs, has become an actual useful tool in a few niche areas for people who understand how to use tools, and that *always* cuts both ways: for better AND for worse. The niches will expand and the technology will merge with older technologies (reinforcement learning in particular) and life will go on. Pandora's box is already fully open (yet again).
When the bubble pops, and hardware prices crash, if the global economy survives, everyone (that cares to) will roll their own LLM rig and can run whatever models they want with whatever "guardrails" they want, or none at all. People will spend years training their own models, avoiding the "dump in everything!" models we have now. Having been trained on mostly good stuff, they'll be actually good at (some) stuff, and might (or might not) become at least sort of actually intelligent. The sky will not fall, the wolves will not eat all the children, and 12-year-olds will (probably) not build nuclear reactors in their backyard. It'll be fine. At least compared to the status quo (which, to be absolutely clear is DEFINITELY NOT FINE!)
The thing is that the incumbent AI companies *desperately* don't want that to happen and they're operating under the delusion that they can prevent it. Most of them are run by a bunch of high-functioning sociopaths and psychopaths (like most global megacorps) who want nothing but more and more money and power, and they see "AI" as the "One System to bring them all, and in the darkness bind them". Spoiler alert: it probably isn't, but I'll carry the metaphor further anyway:
So the question is: will we willingly accept the nine, the seven and the three, and let them rule until they successfully destroy our co-created Human world (which is exactly what they and the rest of the wannabee trillionaire club are doing, whether they know it or not) that they so desperately want to control? I remain hopeful that at least most of us aren't going to blindly accept that, despite literally every day seeing copious evidence to the contrary...
So finally, the real question: will we collectively create a world in which everyone can have their basic needs met, so no one has to *constantly* scramble for survival in this "dog eat dog" world? The answer isn't needing this or that particular socioeconomic system. That's very 19th/20th century. We need to actually come together as a species and tell the handful of psycho rich fucks that run everything and their propagandists to STFU, then just ignore whatever they do, and decide to just actually live our lives.