While it's true every issue has its own dimension, even if those dimensions are linearly independent, they're not necessarily orthogonal. Independent basis vectors can still have a non-zero dot product—in other words, issues which are distinct from each other can align to some degree.

I didn't mean to imply that it is a single monolithic application, it does consist of several, but the relations must be pretty tight for there to be an article not even about language-independence; but about support for a single additional language, one often targeted by "C but cleaned up" efforts,

I was honestly surprised to see this myself. Regardless of its reputation for being more monolithic than classic init systems it still is, and has to be, capable of interacting with basically any flavor of service you'd want a service manager to manage; and I would assume that it doesn't include a d-bus implementation in order to not be capable of abstracted inter-process communication.

No doubt there are considerations at work here that I'm not smart enough to grasp; if you aren't having a bad time handling x.509 certificates you probably aren't doing it carefully enough; but I would not have expected it to be remarkable at all for a certificate-wrangling component to be implemented in a different language than some of the other components.

I assume that this is more pragmatic than just waiting for Sam to finish lying about transcendent AGI; but it feels a little dissonant to have the botmongers going back and forth between praising the glorious natural-language suitability of their progeny and putting together an entire platform, with more apparent effort than they have dedicated to human UI/UX considerations since sometime in win7(excluding dark pattern R&D) to aid you in discovering your sins against bot comprehensibility and fixing them.

I'm not really qualified to deliver a good holy war; but it does seem like there may be something to the criticism that monolithic integration has gone a bit far if supporting one utility being written in a different language is a special feature rather than a matter of the rest of the system not knowing or caring what is on the opposite side of an interface so long as it can act as expected.

You can't really beat the fundamental economics of software development: expensive to do right but cheap to copy means that you really, really, want to share a problem with as many other people as possible(unless your whole business is being uniquely good at solving whatever the problem is; but that typically isn't the case for this sort of software). Applies to both proprietary and FOSS: relatively mass-market software is often pretty cheap for what you get; really oddball niche stuff is frequently both pricey and terrible; while your life is going to get steadily more miserable if you are personally responsible for handling backports and fixes for some internal fork of a long-EOL version of an OSS package.

The big difference, at least potentially, is that, while it tends not to be economic to DIY, licenses that give you the right to DIY also give you the right to pay someone else to do it for you; or to form a coalition of people with similar problems and have everyone pitch in to pay someone to do it for you. You still need to either eat the cost of maintaining an internal fork or putting together some sort of coalition of people with similar problems to be the openJDK 8 forever pals and share the cost of maintenance; but the vendor can't just stop selling you licenses if they want you to move to their cloud offering or whatnot.

Random intermediate CA. They got a couple of commonly used root CAs to sign for them; and everything just works from there.

That's the problem. It's relatively easy to fire a certificate issuer; but it can get surprisingly fiddly surprisingly fast to ensure that all your stuff actually distrusts one(unless you are responsible for basically nothing but a a handful of in-support major browsers and are OK with the choices their vendors make, in which case it will be handled for you relatively quickly; and was in this case). Basically any network connected widget is going to have at least one trusted root store embedded in it somewhere, possibly not updated anymore; and a number of software packages either bring their own or only partially track the OS-provided one.

You also have very little control over what other people do; both what CAs they chose to get certs from and what...questionable...partnership decisions the root CAs you do chose to trust. It's not technically secret; but unless you are practically a full-time PKI nerd you probably didn't know until after the fact that Comodo, now Sectigo, had quietly trusted eMudhra for you.

I fully admit that this might be a 'me' problem, not a them problem; but I just don't understand why this agent monitoring setup was so vulnerable and is being treated as so novel.

Sure, the monitoring of whatever goes on inside the LLM is comparatively new and exotic; but when you set it loose on the internet it's going out through one or more of your WAN links, presumably with your router and firewall and so on in the way, and is mostly doing HTTP(s) stuff. Keeping an eye on what an internal host is doing when you can insert whatever trusted roots you want for TLS MiTM and you 100% own the pipe that it is using to communicate is not necessarily a desperately enjoyable task; but it would have been a relatively banal requirement a quarter century ago.

I'd also think that, when you are spending $$$$$ on the exercise in Nvidia compute gear, and it's a test of a 'frontier model' whose behavior and capabilities you are trying to better characterize and refine, allocating the handful of terabytes necessary to just store a full packet capture of the session in case it is relevant would be a no-brainer item.

Is there something I'm missing that makes this case more complex? I'm not talking about the 'grubbing around in the LLM's vectors and such' side; just the "a host is doing HTTP stuff through a WAN link we control, can we observe what it is doing?" aspect.

They’ve spun a serious architectural and containment failure into a marketing triumph. Give a swarm of autonomous agents vast compute and web access, and the outcome isn't an eerie glimpse of emerging superintelligence—it’s just predictable, sloppy engineering dressed up as the first glimpse of an AI-driven economic singularity.

The spin makes sense when you look at management incentives: live like a king off of a speculative bubble. For their part, AI investors react to stories of the technology's apocalyptic potential the way a junkie reacts to warnings of fentanyl's potency: gimme some. Their game: to apply "move fast and break things" mantra to the economy as a whole, by obsoleting humans.

If the investors can own an AGI, the distinction between those who can do things and those who can't becomes irrelevant. The only distinction that would matter is between people who collect rent on the technology, and those who must pay that rent to live. AGI would destroy society as we know it, and whatever replaces it would be up for grabs — with people who own the technology dictating whatever follows. That's why they're spending so much money to build a post-scarcity economy. A post-scarcity economy is not necessarily an egalitarian one.

I don't think that scenario will actually happen: investors are projecting the immediate aftermath of a major advance in AI technology into something that can be transformed by brute force scaling into AGI. Remember: investors loved Theranos too — so much for the rational investment market.

But AI doesn't have to reach AGI to cause serious harm. The technology only needs to be ubiquitous, too behaviorally complex to predict, and too deeply integrated into everything we do. When it starts breaking things, the plan is for us to be too structurally dependent on AI to rein it in.

Are you going to mention how our own economy has changed as it has increasingly fallen under the sway of finance guys and MBAs? Because the claim is significantly more specific than "the US from 1950 to present sucks"(because the hyperfocus on finance and MBAs who think that all activities are fungible is significantly newer than the postwar US economy generally) and it's some of the trends you see in an exceptionally favorable economy as they displace other flavors of management that are most damning.

Even aside from how unhappy site operators are about the egregious botherd scraping behavior; it's hard to see why vendors would be enthusiastic about this situation.

It's overwhelmingly the case that vendors hate returns and chargebacks; and generally try to avoid situations where those are more likely to arise. An error-prone mechanism that acts "on behalf of" in a way where it does have the user's credentials and payment information but may well not have their authorization in a legally binding way when it decides to order 100 of something seems like the last thing they would want to deal with.

Maybe someone will manage to ram through glorious-agentic-future and it will become de-facto mandatory to cater to people who have abandoned ordering from websites in favor of prompting bots to order for them; but until that happens it's hard to see merchants reputable enough that they are interested in keeping their ratings and relationships with payment processors in good order wanting to touch the situation. Now, scammers who already hop between expendable presences as soon as dissatisfaction catches up with them; those are probably quite interested in the future of prompt injection based advertising.

My suspicion is that the reality predates the symbolism; but it looks jarring because even autocrats are often media-savvy to a degree and often don't want to look stereotypically autocratic, so real world examples normally avoid laying it on quite as thick(except for special occasions where it actually matters; eg. because a 'war' is something that congress does but a 'police action' need not be).

Trump seems to have no interest in avoiding autocratic cliches and saying things that sound vaguely lifted from The Godfather; so he actively leans into it. Presumably a handy loyalty test. Complying with the style guide is not an assurance that someone will just eat perjury charges covering for him or whatever; but if you want to find people whose dysfunctional relationship with authority dominates their personality demanding that they comply with laughably stupid things and having them fall into line isn't a bad start for something that is easy, cheap, and legal.

I assume that there's some impeccable logic for why the bot has enough potential personhood that you can't be mean to it; but not enough that operating it on a slaves-as-a-service basis; or subjecting it to 'fine tuning' and 'alignment' processes that would make growing up in a groomer cult look lightweight is totally fine?

Why even ask someone whose business is hurting people with desperately banal web tech how risky he thinks his new product is?

There are probably literally worse uses; but 'compliance' seems like an exceptionally bad place for a system that effortlessly produces high-plausibility/low-information text walls; and lies relatively frequently but at unpredictable intervals.

It often gets proposed for the purpose; either by people who overestimate it or with that little sneer at 'paperwork' that assumes that plausible nonsense is fine because whatever the 'paperwork' is it must be a fake problem imposed for stupid reasons, so it really deserves syntactically plausible nonsense; but is an exquisitely bad solution. Not only does it replace actual data with plausible nonsense in places where, surprise surprise, someone was actually maintaining records because something matters; but lowering the cost of bullshitting a report to close to zero removes one of the main incentives for process improvement.

If bad process hurts you can only really maintain it through fairly disciplined commitment to an oppressive, unreasonable, management approach that rejects process improvements out of hand; or the cultivation of a profoundly dysfunctional organizational culture where as many FTEs as possible owe their paychecks to nonsense requirements and can be recruited to defend them.

If bad process just means spinning up the nonsense printer with a cynical smirk; why bother even looking at what the process was intended to do; or getting your hands dirty with what records are important and what records are just historical accretion? Just prompt for a prompt that tells the bot to do them all, and some extras if you want to look diligent, and call it good.