Comment Re:Why the 99% confidence interval? (Score 1) 508
A 95% confidence interval is the usual convention, but there's nothing sacred about that number. In my view, it's usually more important to consider expected loss, where you multiply the probability of something going wrong by how much damage would occur for each possible outcome. Let's take a walk and see why you might want a much tighter CI for something like this:
At some point, an automated car will be involved in an accident that causes a fatality. It might be the other car's fault, it might be equipment failure, it might be the AI's fault, but it will happen. People are not very good at understanding risk or teasing out causality, so there will be an outcry calling for heads at Google to roll for this. People will see a new process that takes away control from them, freak out, think that they would do better with their own hands on the steering wheel, and there will be a major push to ban driverless cars. States like California might not give in, since their economy is closely tied into tech companies, but red states? Fuggedaboutit. Emotions trump statistics, and the longer we have to adjust to the idea of driverless cars, the less likely it is that there will be a strong resistance. At this point, we're considering a 5% chance that we were wrong and that driverless cars aren't demonstrably better than humans behind the wheel. A 5% probability of driverless cars being just as unsafe as manually driven cars, multiplied by the lives lost because of delays in implementation after the general public freaks out, is a pretty big expected loss. Truth be told, this probably understates the expected loss, because we're shooting to prove that driverless cars are safer, but it will take a lot of time before we know how much safer. We've been driving for a century, our country is permeated with the car culture, and we have a reasonable grasp of the risks involved (from a statistical, not emotional, level). It will take a while to accumulate the same kind of data on driverless cars.
This also considers a driverless car surrounded by cars with human drivers in the state of California. It's important to keep the scope of the testing in context. It's probably true that we can extend this to the roads in various conditions with varying compositions of driver+driverless cars, but it's easy to see how many people will have reservations about that. Heck, look at this thread.
I can't wait for driverless cars to roll out en masse, not least because of how many lives they will save, but it really is the cultural and legal side that will hold things up across the country. If we have to wait a little longer for the rollout to happen because we're looking for a higher standard of proof, that might be an optimal course of action once you consider that we're dealing with the general public.
Yes, IAAStatistician.