[SystemSafety] Autonomously Driven Car Kills Pedestrian

Sun Mar 25 19:28:17 CEST 2018

David,
You wrote, “On what basis?  ISO 26262 Part 6 is equivalent to the document you’ve referred to although it probably doesn’t have an equivalent to DAL A (don’t forget the A – D sequence is reversed in sense between the two documents).”

I don’t have access to ISO 26262 Part 6. However if it is truly equivalent then it doesn’t really matter which one is applied—as long as one of them is. The reason I singled out DO-178C/ED-12C is that having seen several other supposedly related standards (e.g., medical device development) I consider DO-178C to be a lot more prescriptive in the places that matter. For example, the last time I looked (things might have changed), medical device software development only required “appropriate testing” where “appropriate” is left to the developer's and FDA reviewer’s discretion. DO-178C Level C is much more specific in that it explicitly mandates Decision Coverage. If ISO 26262 is as prescriptive, then I would have no problem using it instead. I would, however, have a problem applying medical device standards because I believe they aren’t prescriptive enough.

The point I was trying to make was that some kind of standard SHOULD be applied, and that standard needs to be adequately prescriptive.

“Also it’s not just about software – what about the hardware it runs on?”

Without a doubt. However, I only feel qualified to talk about the software side because that’s what my background / expertise is. Of course some equally appropriately prescriptive hardware development standard should also be applied.

Regards,

— steve

From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de<mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de>> on behalf of David Ward <david.ward at horiba-mira.com<mailto:david.ward at horiba-mira.com>>
Date: Tuesday, March 20, 2018 at 6:23 AM
To: "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

On what basis?  ISO 26262 Part 6 is equivalent to the document you’ve referred to although it probably doesn’t have an equivalent to DAL A (don’t forget the A – D sequence is reversed in sense between the two documents).

Also it’s not just about software – what about the hardware it runs on?

In terms of Nick’s comments, while ISO 26262 may not have been developed for autonomous vehicles, there is no reason it can’t be used as the basis of an approach.  And yes, there is a requirement in it for independent scrutiny – as an assessor I expect to see a good argument “why” someone has chosen to do something in a particular way, not just “I picked this method because it’s highly recommended in the standard”.  ISO 26262 always requires a rationale for selection of activities not just picking from the tables IMHO.

David

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Steve Tockey
Sent: 20 March 2018 13:19
To: Nick Tudor <njt at tudorassoc.com<mailto:njt at tudorassoc.com>>
Cc: systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

If it were up to me—and it is clearly not—I would require all code for self-driving functions to show compliance with at least Level C of DO-178C / ED-12C before that car can be put on a public road.

My 2 cents,

— steve
发自我的 iPad

On Mar 20, 2018, at 6:15 AM, Nick Tudor <njt at tudorassoc.com<mailto:njt at tudorassoc.com>> wrote:
Hmmm.  ISO26262 is not mandated (SFIK anywhere) and does not cover autonomous cars.  It is also left to the developer to make an argument as to why their systems are safe based upon some subjective activities 'Recommended' etc , by ISO26262.  There is no requirement for independent scrutiny, so a developer can make up any old story and say their product meets ISO26262 (perhaps this is what is going on in the AV world...?) Hence, in my view, it is not particularly 'safe'....and wouldn't add much to the AV world.

Nick Tudor
Tudor Associates Ltd
Mobile: +44(0)7412 074654
www.tudorassoc.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.tudorassoc.com&data=02%7C01%7Cdavid.ward%40horiba-mira.com%7C7730bfe70c9444092f1c08d58e6538c0%7Caa85aed398b34cdab14015ccbb32c3b5%7C1%7C0%7C636571487751412241&sdata=7gIp1BLIcODhq9SnE%2B9gbJila%2FCVjgrY%2FRyRJ1P26bs%3D&reserved=0>
[Image removed by sender.]

77 Barnards Green Road
Malvern
Worcestershire
WR14 3LR
Company No. 07642673
VAT No:116495996

www.aeronautique-associates.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.aeronautique-associates.com&data=02%7C01%7Cdavid.ward%40horiba-mira.com%7C7730bfe70c9444092f1c08d58e6538c0%7Caa85aed398b34cdab14015ccbb32c3b5%7C1%7C0%7C636571487751412241&sdata=SNWEMYC2gQ2%2FO950ZwaCS0icITHoZOAwuG1NtcsMXKk%3D&reserved=0>

On 20 March 2018 at 13:04, Andrew Banks <andrew at andrewbanks.com<mailto:andrew at andrewbanks.com>> wrote:
Agreeing with David (shock horror!) it should also be noted that, in the UK, the official policy of the Department for Transport (DfT) is not to regulate as “this will stifle innovation”, and have explicitly ruled out mandating ISO 26262 (for non-autonomous as well as autonomous) unless and until UNECE do so.

IMHO this policy is very worrying…

A

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de<mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de>] On Behalf Of David Ward
Sent: 20 March 2018 12:36

To: systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

There is an SAE document SAE J3018 “Guidelines for Safe On-Road Testing of SAE Level 3, 4, and 5 Prototype Automated Driving Systems (ADS)” which includes reference to a “Safety development process”.  While this doesn’t explicitly refer to ISO 26262 some of the concepts are very similar.

David Ward

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Martyn Thomas
Sent: 20 March 2018 12:31
Cc: systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

Has anyone actually defined what evidence should be required before a level 5 AV should be licenced?
Regards

Martyn

On 20 Mar 2018, at 12:25, Tom Ferrell <tom at faaconsulting.com<mailto:tom at faaconsulting.com>> wrote:
I agree that legislative bodies should not be waiving standards in general.  However, as I understand it, there are numerous stipulations in place on the self-driving trials underway in AZ including extensive data collection that were put in place by government.  Given the magnitude of what this technology is trying to accomplish, demonstration on closed tracks, through simulation, or controlled experiments is simply not going to be enough.  How long do you propose waiting before these vehicles can be on public streets?

As for the claims relating to lives saved:  This is exactly the same argument that has been made for a host of other ‘safety-enhancement’ systems and devices.  The issue here is that it removes the human from the immediate control loop, a step further than systems like TCAS that ultimately had to be mandated by Congress given the pushback from the airframers and pilots.  The success of that system was quickly evident and is no longer disputed.

I agree that the burden to prove this technology is safe is the designers/manufacturer’s to bear.  I am just saying that we need to stay focused on clearly articulating why we feel the trials or approaches being taken are technically in error or inadequate.  Otherwise, our voices will just get lumped in with the luddites pushing back against change.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of C. Michael Holloway
Sent: Tuesday, March 20, 2018 8:02 AM
To: systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

... This revolution is coming whether we like it or not.  ...
Well, if it should not be coming yet, it is our duty to do everything we can to slow it down to a reasonable pace. That's the point that Prof. Cummings was making in the WaPO article. There are a bunch of ridiculous claims being made by the self-driving car zealots, which are unfortunately being accepted by the US Congress. The most egregious is the assertions about the number of lives that can saved. That's not a reason it is an excuse. If saving lives was the motivation, there are far simpler ways to accomplish it.
What we should be asking is whether the systems being employed in these vehicles have been developed correctly in accordance with ISO26262 or similar standard.
The US Congress exempted self-driving cars from having to meet *any* standards. That should not have happened, regardless of whether this particular accident turns out to have been the automation's fault.
--
All the best,
C. Michael Holloway (cMh)
Senior Research Computer Engineer
NASA Langley Research Center, Hampton VA USA
bit.ly/cmhpubs<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbit.ly%2Fcmhpubs&data=02%7C01%7Cdavid.ward%40horiba-mira.com%7C2472b6f7eed84da3967908d58e5e7b19%7Caa85aed398b34cdab14015ccbb32c3b5%7C1%7C1%7C636571458803641760&sdata=1YRrQ7QbHBR7XY%2FDwFahs0z5RGiPY9C5wKSzAp6K%2B9M%3D&reserved=0>

Verba volant, scripta manent
spoken words fly away, written words remain

(The words in this message are mine alone;
neither blame nor credit NASA for them.)
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>

HORIBA MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 9626352
VAT Registration  GB 100 1464 84

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>

HORIBA MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 9626352
VAT Registration  GB 100 1464 84

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180325/835ff619/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 500 bytes
Desc: image001.jpg
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180325/835ff619/attachment-0001.jpg>