[SystemSafety] a public beta phase ???

[Mike Ellims]

Hi Les,

 

I assume you're aware of the phrase "keep your socks on"?

I know - pot calls kettle black :-)

 

The use of the word "beta" by Tesla appears to be an exercise in moulding
the human perception of the system reliability rather than the fact that it
isn't designed/tested/validated correctly, specifically they appear to be
trying to avoid the perception that it was a perfect finished product.

 

Musk's comment,

 

"Use of word "beta" is explicitly so that drivers don't get comfortable. It
is not beta software in the standard sense."

 

Several things to keep in mind here;

1.       We have no real visibility of what Tesla actually do, there is no
evidence to say either way they are playing fast and loose.

2.       Tesla is under investigation by NTHSA because they asked NTHSA to
investigate, which suggests they have a reasonable amount of confidence they
did the right thing.

 

If we say compare Tesla's response to incidents with say Tepco's; a larger,
more traditional, more experienced safety critical system provider, then
perhaps Tesla isn't doing so badly.

 

It has been suggested (elsewhere)  that Tesla's main problem is perhaps that
they did too well compared with other vehicles that provide similar
functionality (Mercedes, BMW, Infinity) and hence the problems with
maintaining driver awareness as Les suggested.

 

For those interested the following Car and Driver article gives a rundown of
tests they did on several vehicles (and some details of the vehicles
tested).

 

http://www.caranddriver.com/features/semi-autonomous-cars-compared-tesla-vs-
bmw-mercedes-and-infiniti-feature

 

From: systemsafety
[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
Les Chambers
Sent: 14 July 2016 01:18
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: [SystemSafety] a public beta phase ???

 

Hi

"light thickens and [as] the crow makes wing to the rooky wood ..." I have
grave fears for the state of driverless car development at Tesla.

Frankly, the first paragraph in this article from IEEE spectrum has blown my
small mind!

"The first death of a driver in a Tesla Model S with its Autopilot system
engaged has exposed a fault line running through the self-driving car
industry. In one camp, Tesla and many other carmakers believe the best route
to a truly driverless car is a step-by-step approach where the vehicle
gradually extends control over more functions and in more settings. Tesla's
limited Autopilot system is currently in what it calls "a public beta
phase," with new features arriving in over-the-air software updates."

PUBLIC BETA PHASE!!! Jesus wept are these people serious! The subtext here
is silicon Valley 20 something phone app developers programming life
critical systems where a "blue screen of death" is no longer a giggle but
means real death. Buy a Tesla and your life is in the hands of a BETA
release? WHAT?

 

".. good things of day begin to droop and drowse ... "

 

Over there in Europe we have committees of good men and true developing
standards for safety critical systems aggregating the best of what we know
about building safe systems.

 

" ... While the night's black agents to their prey do rouse ... "

 

While in silicon Valley we have an ex PayPal developer ignoring the lot,
turning the world on another axis so to speak ... that of agile rapid
development ... "Hey I have this neat feature ... oh sorry you died ... But
remember we told you to be careful ... You're the driver in the loop ... (or
you were) "

The interesting thing is that this approach will accelerate the development
of this technology, but not without casualties, which is okay as long as the
casualty is not you or I.

The sad thing is: any experienced automation engineer knows that depending
on a human in the loop to behave rationally in an emergency is rank
stupidity. And any death caused by engineering stupidity is a death we must
avoid.

This scenario smacks of the Armagh rail disaster (1889), the one that
heralded the first regulation of railway networks - where the regulators
stopped making suggestions and started throwing people in jail for
non-compliance. 80 people had to die (a third of them children) to make that
happen. I sincerely hope that we don't have to watch any children die before
US regulators force Musk to put some adults in his development shop.

 

"... Thou marvel'st at my words: but hold thee still.

Things bad begun make strong themselves by ill. "

 

Les

 

 

-------------------------------------------------
Les Chambers
Director
Chambers & Associates Pty Ltd
www.chambers.com.au

Blog: www.systemsengineeringblog.com
<http://www.systemsengineeringblog.com/> 

Twitter: @ChambersLes <http://www.twitter.com/chambersles> 
M: 0412 648 992
Intl M: +61 412 648 992
Ph: +61 7 3870 4199
Fax: +61 7 3870 4220
les at chambers.com.au
-------------------------------------------------

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160714/ba42d4ee/attachment.html>