[SystemSafety] Autonomous Vehicle Safety

[Phil Koopman]

Going by memory, the free RAND report has the same or similar material 
as the paywall article.

This is a dated paper, and since then there has been much exploration of 
the limits to such a calculation, which I will enumerate as threats to 
validity to a conclusion that such vehicles are acceptably safe:
- Every (weekly?) software update potentially invalidates previous 
experience
- Every continual change to the operational environment potentially 
invalidates previous experience
- The baseline rate for human drivers has quite high variability 
depending on numerous factors and is difficult to pin down
- There is incentive for autonomous vehicle companies to cherry-pick a 
baseline (e.g., only "more than 51% at fault crashes" or only "paid 
insurance claims" or comparing low-risk operational conditions to 
averages that include high-risk conditions)
- There is a much more to safety than net statistical failure rates, and 
in particular various flavors of risk hot-spots matter significantly for 
real-world acceptance (e.g., risk transfer, negligent driving behavior, 
trust-degrading behavior).

For those who have missed the social media announcements, I have a new 
book on Embodied AI Safety which goes into these issues that builds on 
my previous How Safe Is Safe Enough book. I have a pair of free talks as 
well that dive quite deep into these topics.
   - How Safe Is Safe Enough: https://youtu.be/UTdR_HE3DDw
   - Embodied AI Safety: https://youtu.be/yk0iGRzOLD4

Happy holidays,
-- Phil

On 12/17/2025 10:13 AM, Prof. Dr. Peter Bernard Ladkin wrote:
> Bruce Schneier's latest CryptGram points to a 2016 paper answering the 
> question of how many driving miles it would take to demonstrate 
> autonomous vehicle safety. Given that there are some people on this 
> list who have been dealing with such calculations for decades, I 
> thought it worth sharing the link and asking for comments.
>
> I haven't read the paper. Elsevier wants $9.50 for 24 hours rental and 
> $55.20 for purchase. If you are affiliated with an institution that 
> subscribes to Elsevier packages, maybe you can get it at no cost (to 
> you).
>
> https://www.sciencedirect.com/science/article/abs/pii/S0965856416302129
>
> On the other hand, since the first author is at RAND, maybe there is a 
> report to download at no charge. Indeed so, through
>
> https://www.rand.org/pubs/research_reports/RR1478.html
>
> PBL
>
> Prof. Dr. Peter Bernard Ladkin
> Causalis Limited/Causalis IngenieurGmbH, Bielefeld, Germany
> Tel: +49 (0)521 3 29 31 00
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: 
> https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
>

-- 
Phil Koopman    m: 412-260-5955    <phil.koopman at hushmail.com>