So last week's "summit" went that well, eh?

So we're going to use an AI to monitor AIs to make sure they're not doing anything naughty? Pretty sure I've seen that movie already...

Not to mention, who exactly pays for Sentry to run? Golly, if some completely random AI hardware manufacturer were to somehow profit off designing and selling a scheme to waste extra tokens on yet another layer of supervision, that would almost look just a wee bit unethical!

That wasn't quite the response I expected - I guess I'm not entirely clear on whether you were mocking those guardrails, or presenting them as legitimately (albeit perversely) meaningful.

Either way, my central point was "why not both". It's entirely possible to be neutral, in the sense of calculating the mean (in the internal thought-vector sense) of all available viewpoints. If, however, 99% of sources are talking positively about some trashy conspiracy theory - Any version of "consensus" is itself going to lean pretty far into wingnut territory. In that regard, I think we might actually be agreeing?

We can meaningfully ask the question "For each location a robotaxi has been involved in some sort of incident, how does the incident rate for both humans and other robotaxies comapre?"

That's not a meaningless number, and it's entirely possible Waymo really is doing better than humans - But only in situations Waymo gets to cherry-pick simply by enforcing their designated service areas. Meanwhile, let me know when a Waymo can successfully navigate overgrown single-track dirt roads common in my area. Anyone know offhand if they have moose in the Philippines?

I'm a fan of drastically slowing AI development until we've solved the alignment problem, but a kill switch ain't it.

What would a kill switch really do? An AI is fundamentally nothing more than a description of it's topology and a set of a few billion weights. Implementing that topology and running those weights is the easy part and any coder worth a damn could roll their own (performance aside) with minimal effort - without bothering with a kill switch, if they so choose.

The only way to even come close to enforcing a kill order would be for every sufficiently-performant processing platform to enforce some kind of blacklist of known-evil weights in a way that's resilient across different quantization levels. And even then, simply running pre-killswitch drivers would still be a trivial way to circumvent any such restrictions entirely. Or for the truly determined, you can run even top frontier models on vastly underpowered hardware like a mid-range desktop CPU. Sure, your throughput might be measured in minutes per token, but realtime feedback isn't going to change the answer to "how do I hack the Gibson". An accurate answer a few days from now is likely to be just as useful in the vast majority of situations.

That all assumes there are more-or-less equally relevant answers available from both trusted and biased sources.

If someone asks about Hunter Biden's penis, that assumption is false, and the vast majority of relevant, if not necessarily accurate, information will come from partisan shills. There aren't any noteworthy safety issues, aside from the abstract risk of a third term leading to our next civil war. And there isn't any middle ground, since the response to the right's delusions isn't rebuttal but rather, a pretty much uniform "what the fuck is wrong with you people?" from the left.

We always need to remember AI isn't "smart", it's nothing but autocorrect on crack. It can't meaningfully evaluate those criteria we're giving it in any way except shifting the probabilities of multiple competing matches. If there's only one viewpoint represented in vast majority of training data, that's the one it's forced to choose.

I asked my buddies ChatGPT, Copilot, Gemini, and grok and they all said they would never lie to me!

Who's at fault in those crashes?

Hint: a lot of crashes with fully self driven vehicles is because they actually obey traffic laws in places where locals are used to not doing so.

Smells come from antisocials. Normal people wash and wear mild deodorant (if not east asian, in which case they just wash).

Usage of public transit is fundamentally limited by antisocial people.

No one wants to be on the same bus as a black guy going "I'm going to kill this white girl" and then stab you in the neck. No one wants his wife, daughter, mother, etc to be Iryna. No one wants them to be that girl that got burned alive either. Or any other of the countless public transit statistics.

That is the problem with public transit, and that's why it's usage is now suffering in Europe as well.

So that is a whole different problem, completely unrelated to the problem of self driving. Self driving primarily replaces one's own driving. It's not for the people so poor, that they can't escape the antisocials.

But you can't, because Waymos drive drunks and octogenerians. That means you have to factor them in in a world where both are banned from driving, but drive anyway because "human factor".

Just like you have to factor in the "AI factor" with Waymos, such as AI just having a momentary aberration at a critical moment. It's just that there's a whole lot more of A than B.

I suspect this is because Waymo uses a narrow very high definition model of its driving environment. I.e. they are really, REALLY good at areas they know, but it's a lot of effort to expand the area where they are good.

It's the opposite approach of "wide lower resolution" model that Tesla uses. This is why Tesla's FSD works in most places in Western nations, but it will probably have more problems.

Tesla FSD basically offers you a mode to break the law, because in many areas, breaking traffic laws is a norm and it's actually less safe to be the only one following the speed limit, not overtaking agressively and so on.

The idea is that most drivers are aware of local driving culture, and can adjust as needed. You can be in Mad Max mode in Miami, and strictly law abiding in a German village.

The punishment is that since its FSD supervised, any punishments from selecting wrong kind of mode and letting it stay on as you observe it driving against local driving culture is that you are responsible for fines. Just as you would have been if you drove like that yourself. Just like any manager supervising things that do wrong things under your supervision.

They are in cities. Waymos don't drive highways all that much to my understanding.

And yes, humans are very good on average. But difference between (for example, number for illustration purposes only) 99.9% good and 99.99% good when it comes to millions of crashes every year is a meaningful number.

I don't think there is a solution. In all seriousness, if I were starting my career from scratch today, I would avoid anything that can be done purely remotely, and quite likely go into a skilled trade - Not romanticizing "hands-on" work, I realize it has its own down-sides, but it's becoming increasingly clear we'll still have elevator techs and plumbers longer than we'll have programmers and accountants.

AI has effectively already killed white collar jobs going forward, whether or not those outside tech have noticed yet. It can more-or-less replace junior-level workers today, and with no juniors there will be no seniors later and no principals twenty+ years from now. Though even if we somehow find a way to backfill the experience pipeline, I have little doubt AI will be coming for even the best of us within a decade. Maybe it'll hit some unforeseen ceiling before then, but that's not a bet I'm eager to take.