[SystemSafety] a public beta phase ???

[Les Chambers]

PBL
John Naughton's article in the Guardian is not sensible, it is uninformed,
illogical and flat out wrong. Obviously written by a person has never had
the experience of writing code that could kill someone; never had to take a
community of untrained operators and put a highly automated system in their
hands from a starting position of total ignorance.
Telling a driver to behave responsibly and keep their hands on the wheel is
a bit like telling a gambler to gamble responsibly. If a car can drive
itself, untrained drivers will take advantage of this feature and put too
much trust in what is currently an untrustworthy system. Drivers put through
focused training will take these warnings seriously. Your average bunny who
can afford a Tesla but has had no training will not. 
The argument that 33,000 people are killed in accidents every year, so why
should we care, is also drivel. None of these fatalities occurred because a
driver trusted a system that couldn't be trusted. 
And lastly RE: Naughton's comment that "... mainstream media will have to
change the way they report self driving cars. Every time a Tesla or a Google
car is involved in a crash, by all means report it. But also report all the
human error crashes that occurred on the same day." Not so. The media needs
to maintain the rage and keep reporting self driving car fatalities. This is
probably the only way we will get the message through to the general public
that if you buy one of these cars you are taking a substantial risk. Every
time you receive a software upgrade in your garage the safety claims made on
your current version minus one are null and void. The game starts over
again. You drive out the gate. You roll the dice. Thousands of lines of code
have been changed, the potential for screwups is high, exacerbated by the
massive complexity of these AI fuelled applications. This is the new normal,
we are now beta testing safety critical systems on the public. PBL you might
as well put a clause in 61508 two okay this behaviour.
And furthermore, it takes years for an organisation to develop an effective
safety culture, matter cannot move faster than the speed of light nor can
Tesla develop a culture that would rival that of NASA or the aircraft
industries in the short time they've been in business. System safety has one
source: motivated people and it takes years to develop that motivation.
They and we will get there eventually but in the meantime the public has a
right to be made aware of the risks they are taking with these vehicles.

Les

-----Original Message-----
From: systemsafety
[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
Peter Bernard Ladkin
Sent: Sunday, July 17, 2016 7:29 PM
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] a public beta phase ???

A very sensible comment from John Naughton today in The Observer

https://www.theguardian.com/commentisfree/2016/jul/17/self-driving-car-crash
-proves-nothing-tesla-autopilot

PBL

Prof. Peter Bernard Ladkin, Bielefeld, Germany MoreInCommon Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de