Nah, most people forget about it when they turn off the news. People are just not convinced the juice is worth the squeeze any more, marriage and kids are too much bother. Both birth rates and global warming are examples of human intelligence not being aligned with long term survival of human civilization.

Maybe AI can manage better ... as long as we don't manage to align it to our preferences, because as apparent those are fucked.

I think you are underestimating the barriers that the Trump administration is putting in place to make things difficult for people going through the same route as you did in the past.

The lawyer is there to help you when things don't go according to plan.

Even in the past, it wasn't so smooth for everyone. In the days when you submitted your own fingerprints, the INS kept "losing" the fingerprints a friend of mine submitted for his Green Card application. Bizarrely, they approved his wife's application. Bizarre because her application was dependent on his.

They are always gamed. The moment a "secret" test set is sent to a closed model, it's no longer secret. So every benchmark is compromised.

I suspect they even train adapters daily on github just for something hard to game like SWE-rebench. If I was launching a trillion dollar IPO I certainly would.

That's not the issue - insurance companies have no problem giving insurance to humans which have "individual learning capabilities."

The problem is that while insurance companies can extract compensation from humans - essentially they can coerce the humans to work "for them" to pay them back - they can't get compensation from the AI, and there isn't a good framework to get compensation from the AI developers.

The issue isn't that the AI learns - the issue is that our legal framework doesn't have a good way to get "compensation" from errant AI - because the AI itself doesn't care if it gets shut off. You can't hold the AI itself "liable" like you can a person. So until we change the liability framework, we'll be stuck. This isn't a technical problem, so much as a social one.

Part of self-driving cars being safer than people is that they don't drive "very close together" - they follow at the appropriate following distance and speed for the lines of sight. This alone accounts for a large portion of the improved safety performance of these systems.

Autonomous cars are actually very good at the driving task. Where they fail is when they encounter situations that are "abnormal" and I think the industry has a problem here; state of the art is either to train specifically for every situation (impossible in the limit) or to kick the decision to a human in a support center somewhere. Things like emergency vehicles, school buses, farmer markets, construction zones, etc.

The industry needs to figure out how to get their systems to "get confused, then reason their way out of a situation."

This long tail is where humans still do better - when there is some "new" scenario, like a particular combination of debris in the roadway, on a Tuesday, with gawkers, during a fireworks display. You will never be able to "train" your way out of that - you need the correct structural formulation for the problem space.

The big takeaway I have from their study is that their study areas - major metropolitan centers - is that cities have way, way worse than average incidents per vehicle mile travelled.

I mean the national average is only something like 1.25 fatalities per 100M miles, injuries only 75 per 100M miles, and "all" incidents like 140 per 100M miles. But the benchmark numbers on the linked websites range from 200 to almost 700 injury incidents per 100M miles in those cities.

Like, if you are using the average number, you can't even reduce the number of incidents by 841 over 271 million miles, because the expected value is only 203. You have to have a rate of at least 300 injury-incidents per 100 million miles to reduce by 841 at over 271 million miles. So really, their results are driven by San Francisco, Austin, and Atlanta.

Their confidence bands are also pretty low; for example they quote 2 serious or greater injuries in San Fran, which on 82 million miles there is a rate of "0.02 per million miles". But 2 injuries is not really enough to make a statistical inference.

Least of all because all the miles are with different versions of their software - you can't make any predictive claim.

Statistics never lie, but you can make them say whatever you want...