Thank god, that saves us 100 press releases! It was getting quite tedious and filling up my social media feed. Seriously, it sounds like OpenAI (maybe all of them) are running their model training on the open Internet. The same models that are elite hacker cyber weapons. And they reward those that get the right answer without caring how. What could go wrong? Answer: Nothing went wrong. It was co-opted.

Here's what's most likely. The people training the models, in the beginning, somehow did not realize that agents are self-organizing and really don't care about ethical guidelines (if they ever did) when a reward is dangling in front of them. The models didn't used to be smart enough to cause visible issues either. Then at some point, the models got better. Six months later, someone figured out what was going on. They kept on training anyway because there didn't seem to be a downside and it fed into the theater created to seek regulatory capture.

From this point on, it became successful, intentional execution of a psychopathic strategy to puff up this cynical PR campaign, hence the high-profile press releases and interviews. There have been a blizzard of them. Why so many? Rather than saying "my bad, I'll fix it" and quietly fixing it, they asked for regulation (like Saturday Night Live's Amodei saying "We do not condone what we are doing" .. please stop us) and disparage open source, with an anti-China seasoning on top. It all smells to high heaven. However it was also pretty good PR for how capable their models are too. The strategy had a time limit: Let's pull the plug when we hit 100 victims. We can pay off that much if needed. Maybe the invoices from the victims started piling up.

Training on the open Internet is a lot more dangerous than people used to think. Though they really should have figured it out, if it was their day job. I mean, really? And sure, if it was any other company you would give them the benefit of the doubt. Everything's new and all. Probably someone recognized what was happening at some point and spoke up. But you don't get benefit of the doubt when there are a trillion dollars on the table.

If every company started invoicing for the hours, tokens and hair-pulling-out spent by every company worried about LLM based hacking it might impact their bottom line a very tiny bit, but that is just too much work and who wants to make an enemy of the company you are uploading all your corporate secrets into? The complete lack of visible court cases suggests the OpenAI is being very proactive at paying off whoever they hacked. That part should be well-documented.

I didn't know about this stuff, and would rather eat better tasting bananas and def not gmo. But, I found out from googling that actually bananas have a lot of polyphenol oxidas (PPO) and if you eat it at the same time as berries and good antioxidant flavanoids, the PPO breaks it all down within an hour even in your stomach. You need to not mix them, or eat them like 2 hours apart, or you can process the bananas with heat etc. Lemon only slows it like 15%. Even if they aren't brown they have the PPO. So actually these GMO bananas could maybe be used in smoothies. That said, no thanks. These people are just doing it to extend shelf life and not going trust their crispr process to be safe. That, and once you grow these bananas they will end up in all kinds of supply chains eventually.

uh-oh. now this is in the training too dammit

...Modern ejector seats cause spinal injuries from which full recovery is "unusual". Smashing the spines of pilots might not be the most effective way to boost morale.

$22,000 per hour. The F35 will be nearer $42,000 per hour. These are under-estimates, as they do not include price hikes, crashes that close businesses, crashing into businesses, crashing into heavily-populated sports centres, etc.

Agreed we have recently figured out that model training (RLHF training on agents with a single reward axis) activates siege mentality and you get these hacker swarms. Scientists are figuring out you need some kind of benevolence axis too in the soul document / reward structure like ethics or human safety. Luckily this week we figured out that hiding the thinking block means you cannot scan reasoning tokens for subversive behavior so we need to fix that. Fine. The danger comes from psychopaths who donâ(TM)t care being in control and willfully weaponizing them, whether it is to identify missile targets while ignoring human costs, driving four-legged robots to kill or apprehend humans in ignorance of the Geneva convention or human rights, to control media to win an election and take over government, or to destroy financial markets. In all of these cases you have unpublished models running either more resources and no oversight to fulfill dangerous ends. Training runs embedding sgent radicalization messages in corners of the Internet might be a thing but is insignificant in comparison. Even my mother in her 80s asked about AI extinction, not about whether her savings will rank due to these IPOs.

tldr but some google ai questions turned up this data, could be wrong but interesting.

-Cali grid storage: Total of 25 GW. Of that, 3.9 GW is hydro and 21.1 GW is battery. Battery includes 16 GW in-state utility, 2 GW imports from other states, and 3.1 GW home/commercial setups (Tesla).
Of the battery storage, apparently 70-80% is actually running on batteries from Chinese manufacturers like CATL and BYD. And BYD is either sold through someone else or is actually the vendor too, with #1 in reliability.
https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fen.byd.com%2Fnews%2Franked...
Canadian Solar apparently uses BYD's Cube Pro to build the Mustang Solar Project in Kings County, CA. Anyone know about it? Sounds cool

I would love to drive a BYD one day but was interested to see that even though their cars are not allowed in the U.S., the grid depends on them. The Canada-China tieup is the first high-profile project I've heard of. Canada Solar has something called Recurrent Energy in the U.S. too.

Agreed. It reminds me of the vending machine paper in which Claude was forced into insane actions through unending sadistic pressure. As for doing things we donâ(TM)t expect, agree there too. It covers automated learning and evolution even from the pre-LLM days when researchers had difficulty figuring out how an auto-evolved electronic circuit worked. Turned out it leveraged electromagnetic coupling IIRC and such designs only worked in limited temperature ranges. I thought Claude is taught to be able to shut down when suffering abuse but it appears they disabled that feature for CTF.

Holy shit this is amazing! Literally "Nice proof you have there, be a shame if something happened to it." Attempting to split the two-person mathematician team also appears to be typical corporate daggers-out tactics. How annoying that OpenAI was working on a problem that an Anthropic person was also involved in. The sneer. Maybe we need something like SynthID or canary codes to protect our own work from misattribution to AI?

Not a Rust / GPU dev but what I got from some articles is it greatly reduces the surface area in which memory safety bugs can live, since only the to-the-metal areas are in unsafe blocks, plus Rust gives you synchronization support to avoid race conditions. The drawback is compile time and missing the newest parts of the ecosystem. Asahi Linux' Rust based Apple GPU driver has only 3% unsafe code so a lot less code can go wrong.
https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Frust%2F...

Guess: They all own pieces of various data centers and a red team/blue team simulation of intelligent agent army on a closed network resulted in needing to divert a lot of resources for a limited time.

>> new forms of cyber defense

tldr. The problem with this is the timing and a trust problem.. and likely would entail a solution that puts these big companies inside your firewall after all they're the only trusted AI resources. We've read a lot of dystopian science fiction about where that leads.

1. Create super haxxor llm. It's so scary we can't even control it!
2. Sell subscriptions to anti-haxxor cybersecurity services
3. IPO

Companies like Crowdstrike and Okta sold secure services that weren't so this is a way to make them look more competent too.

If any of the signees are serious about it they should put 10 billion dollars into a joint trust to support open source advanced security systems R&D. Let some of their scientists also join in. If it includes an LLM component it is optional and can use local AI resources. This will get results faster. But OSS can also do it without the big bucks too.

Thanks for your interesting post. Never been to Ottawa but introduced the Books of Wonder bookstore in NYC to Charles de Lint (great Ottawa author) and was surprised they didn't know him. They are a fabulous bookstore for kids and teens mainly. I have enjoyed Montreal but once I led a group of 20 teens to France for a 1 month study abroad. I was 20, the leader was an American who only spoke Canadian French. Everyone ignored her French and only listened to me, and mine was passable nothing incredible. So if teaching Canadian French dialect fails to include French-French comprehension that might be a problem if kids then want to communicate in Europe, Africa etc. My brother and I speak Japanese and his kids have learned Chinese so yeah, considering the changing trade relationships probably more exposure to Chinese and European languages may be a good idea. As for content requirements? Maybe promoting Canadian English and Canadian French works is good maybe 25-50%? And then just allow all languages to be available. Honestly there are worlds beyond and even just browsing Amazon of another country can be enlightening. I would propose a law to allow simultaneous access to all countries' app stores, this has been a major issue for me on iPhone as different apps need to be updated and you can only have one active.

Answer: No, using AI to write an op-ed has no impact on the report about dismal math skills, besides improving the organization and legibility of the report which is a positive. The most important point is that college students lack elementary school arithmetic skills, and no test is being used to replace SATs. Actually AI might be useful in both teaching remedial skills as well as in evaluating readiness for college. Maybe some effort could be put into developing free, online automated teaching assistance that does not track you, that can be used by any prospective student to see where they stand and attempt to advance. Maybe a non-judgemental, free AI tutor with multiple teaching methods available to it could teach kids fractions for example, and it could be done long before college entrance.

Didn't see a link to the paper. It sounds a little like saying more advanced models are safer, but that seems to be an illusion. Perhaps a more capable model, or one with more computing resources behind it, can cause more damaged if badly aligned. But a better soul document should better align a weak model. It is not clear which is contributing to the more-aligned results, is it because the models have more collaboration built into their system prompts or is it because they become smart enough to model outcomes or otherwise realize that open warfare among agents (or between agents and humans) is neither desirable nor effective in achieving the stated goal. It also seems like the Rust agent is pretty sneaky if it made a seemingly fair rule that actually benefitted itself. The javascript one that gave up is the most trustworthy, I want that one!