[SystemSafety] a public beta phase ???

[Les Chambers]

Hi

"light thickens and [as] the crow makes wing to the rooky wood ..." I have
grave fears for the state of driverless car development at Tesla.

Frankly, the first paragraph in this article from IEEE spectrum has blown my
small mind!

"The first death of a driver in a Tesla Model S with its Autopilot system
engaged has exposed a fault line running through the self-driving car
industry. In one camp, Tesla and many other carmakers believe the best route
to a truly driverless car is a step-by-step approach where the vehicle
gradually extends control over more functions and in more settings. Tesla's
limited Autopilot system is currently in what it calls "a public beta
phase," with new features arriving in over-the-air software updates."

PUBLIC BETA PHASE!!! Jesus wept are these people serious! The subtext here
is silicon Valley 20 something phone app developers programming life
critical systems where a "blue screen of death" is no longer a giggle but
means real death. Buy a Tesla and your life is in the hands of a BETA
release? WHAT?

 

".. good things of day begin to droop and drowse ... "

 

Over there in Europe we have committees of good men and true developing
standards for safety critical systems aggregating the best of what we know
about building safe systems.

 

" ... While the night's black agents to their prey do rouse ... "

 

While in silicon Valley we have an ex PayPal developer ignoring the lot,
turning the world on another axis so to speak ... that of agile rapid
development ... "Hey I have this neat feature ... oh sorry you died ... But
remember we told you to be careful ... You're the driver in the loop ... (or
you were) "

The interesting thing is that this approach will accelerate the development
of this technology, but not without casualties, which is okay as long as the
casualty is not you or I.

The sad thing is: any experienced automation engineer knows that depending
on a human in the loop to behave rationally in an emergency is rank
stupidity. And any death caused by engineering stupidity is a death we must
avoid.

This scenario smacks of the Armagh rail disaster (1889), the one that
heralded the first regulation of railway networks - where the regulators
stopped making suggestions and started throwing people in jail for
non-compliance. 80 people had to die (a third of them children) to make that
happen. I sincerely hope that we don't have to watch any children die before
US regulators force Musk to put some adults in his development shop.

 

"... Thou marvel'st at my words: but hold thee still.

Things bad begun make strong themselves by ill. "

 

Les

 

 

-------------------------------------------------
Les Chambers
Director
Chambers & Associates Pty Ltd
 <http://www.chambers.com.au> www.chambers.com.au

Blog:  <http://www.systemsengineeringblog.com/>
www.systemsengineeringblog.com

Twitter:  <http://www.twitter.com/chambersles> @ChambersLes
M: 0412 648 992
Intl M: +61 412 648 992
Ph: +61 7 3870 4199
Fax: +61 7 3870 4220
 <mailto:les at chambers.com.au> les at chambers.com.au
-------------------------------------------------

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160714/d1fd5bcc/attachment.html>