[SystemSafety] Autonomously Driven Car Kills Pedestrian

[Michael Jackson]

Martyn, 

In your very nice lecture about driverless cars you rightly express (page 5, last paragraph) 
your doubt that an overseeing human driver could reliably take control when the automatic 
system fails or appears about to fail. You also point out the well-known evidence that pilots 
similarly cannot take over reliably in an emergency. I offer a supporting point, and also an 
anecdote. 

First, 'becoming alert and aware’ is not an atomic instantaneous event: it takes time. Worse, 
a competent human driver’s awareness of the traffic situation at a particular time is based 
partly on observations made at earlier times. If you see a cyclist beginning to push her bike 
across the opposite carriageway you should be fully prepared for her to continue across the
division into the carriageway on which you are driving. 

Second, a true anecdote. A driver driving on an urban dual carriageway in light traffic set 
cruise control to the speed limit of 60mph. He then took his feet off the pedals and placed 
them with the soles flat on the carpet. A little later, suddenly seeing that he was coming to 
a junction showing a red light, he tried unsuccessfully to brake, found that his efforts had 
no effect, and crashed hard into the back of a car stationary at the lights. It turned out that 
he had been pressing the clutch, not the brake. (Fortunately no-one was seriously injured.) 

For me, the moral of the point and the anecdote is that when you are ‘relaxing' while the car 
drives itself you must pay full attention to the traffic situation at all times, and also keep the 
positions of your hands and feet, and your general body posture, constantly at the ready to 
take over instantly. Nothing less will do. 

So the responsibility in overseeing autonomous driving is worse than that of an old-fashioned 
driving instructor in a dual-control car, teaching an untrusted learner—you can’t even order 
the software to slow down: in short, it is far more demanding and stressful than driving the 
car yourself. 

Regards, 

—Michael 
   

> On 23 Mar 2018, at 14:21, Mario Gleirscher <mario.gleirscher at tum.de> wrote:
> 
> Thomas,
> 
> On 23.03.2018 10:41, Martyn Thomas wrote:
>> On 23/03/2018 10:11, Peter Bishop wrote:
>> 
>>> There are lots more non-fatal accidents, and this may one way of getting
>>> an earlier estimate of how well self-drive is doing - i.e. are the
>>> non-fatal accident rates similar? Not sure what the human accident rate
>>> is, but my guess would be 1 in 100000 miles, so you could get 95%
>>> confidence for that rate with around a quarter of a million miles of
>>> accident free self-drive.
>> To compare like with like, you need reasonable confidence that the mix
>> of hazardous situations is the same, and that the reporting and
>> classification of "accidents" is the same, and that what you are
>> measuring is a strong indicator for what you really care about.
> 
> Absolutely right, and I don't know of efforts to standardize driving
> situations. I remember an interview with a senior safety engineer from
> car industry telling me that driving scenario taxonomies are typically
> even specific to the company's engineering sub-department they are used
> in, although, what I understood from this persons statements, this is
> except for some local details not necessary and could be very well
> harmonized, be it only within an organization.
> 
> Knowledge about driving situations has to be open, it can not
> intellectual property, by no means, ever.
> 
> Moreover, that's why I tried to point at the problem of looking behind
> the curtains of the field test procedures and clarify questions across
> car vendors: which driving scenarios where actually tested in which way
> exactly, under which conditions, how often, how were test results
> compared in e.g. "shadowing experiments", etc. etc. Surely, the
> combinatorial space of such test suits would blast any company's budget
> multiple times. Though, we as a society need to have the chance to take
> part in the decision process on the sub-spaces of the whole test space.
> And I am not only talking about "safe and ethical driving on open
> streets for machines" like in upcoming IEEE standards.
> 
>> To illustrate the last point: it would be important to know if AVs had
>> far fewer cosmetic accidents (scrapes) but more accidents that caused
>> serious injuries.
> 
> My Friday afternoon 2p.
> 
> Have a nice weekend,
> Mario
> 
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE