[SystemSafety] a public beta phase ???

[Matthew Squair]

Words are very important, calling something an 'autopilot', rather than 'driver assist' is probably a mistake. Using the term 'beta' to engender a degree of caution in the user base is, conversely, probably a good idea. Thinking about it in terms of risk homeostasis, Tesla appears to be trying to put a big virtual spike in the center of their steering wheel :)

Matthew Squair

MIEAust, CPEng
Mob: +61 488770655
Email; Mattsquair at gmail.com
Web: http://criticaluncertainties.com

> On 14 Jul 2016, at 8:11 PM, Mike Ellims <michael.ellims at tesco.net> wrote:
> 
> Hi Les,
>  
> I assume you’re aware of the phrase “keep your socks on”?
> I know - pot calls kettle black :-)
>  
> The use of the word “beta” by Tesla appears to be an exercise in moulding the human perception of the system reliability rather than the fact that it isn’t designed/tested/validated correctly, specifically they appear to be trying to avoid the perception that it was a perfect finished product.
>  
> Musk’s comment,
>  
> “Use of word "beta" is explicitly so that drivers don't get comfortable. It is not beta software in the standard sense.”
>  
> Several things to keep in mind here;
> 1.       We have no real visibility of what Tesla actually do, there is no evidence to say either way they are playing fast and loose.
> 2.       Tesla is under investigation by NTHSA because they asked NTHSA to investigate, which suggests they have a reasonable amount of confidence they did the right thing.
>  
> If we say compare Tesla’s response to incidents with say Tepco’s; a larger, more traditional, more experienced safety critical system provider, then perhaps Tesla isn’t doing so badly.
>  
> It has been suggested (elsewhere)  that Tesla’s main problem is perhaps that they did too well compared with other vehicles that provide similar functionality (Mercedes, BMW, Infinity) and hence the problems with maintaining driver awareness as Les suggested.
>  
> For those interested the following Car and Driver article gives a rundown of tests they did on several vehicles (and some details of the vehicles tested).
>  
> http://www.caranddriver.com/features/semi-autonomous-cars-compared-tesla-vs-bmw-mercedes-and-infiniti-feature
>  
> From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Les Chambers
> Sent: 14 July 2016 01:18
> To: systemsafety at lists.techfak.uni-bielefeld.de
> Subject: [SystemSafety] a public beta phase ???
>  
> Hi
> "light thickens and [as] the crow makes wing to the rooky wood ..." I have grave fears for the state of driverless car development at Tesla.
> Frankly, the first paragraph in this article from IEEE spectrum has blown my small mind!
> "The first death of a driver in a Tesla Model S with its Autopilot system engaged has exposed a fault line running through the self-driving car industry. In one camp, Tesla and many other carmakers believe the best route to a truly driverless car is a step-by-step approach where the vehicle gradually extends control over more functions and in more settings. Tesla’s limited Autopilot system is currently in what it calls “a public beta phase,” with new features arriving in over-the-air software updates."
> PUBLIC BETA PHASE!!! Jesus wept are these people serious! The subtext here is silicon Valley 20 something phone app developers programming life critical systems where a "blue screen of death" is no longer a giggle but means real death. Buy a Tesla and your life is in the hands of a BETA release? WHAT?
>  
> ".. good things of day begin to droop and drowse ... "
>  
> Over there in Europe we have committees of good men and true developing standards for safety critical systems aggregating the best of what we know about building safe systems.
>  
> " ... While the night's black agents to their prey do rouse ... "
>  
> While in silicon Valley we have an ex PayPal developer ignoring the lot, turning the world on another axis so to speak ... that of agile rapid development ... "Hey I have this neat feature ... oh sorry you died ... But remember we told you to be careful ... You're the driver in the loop ... (or you were) "
> The interesting thing is that this approach will accelerate the development of this technology, but not without casualties, which is okay as long as the casualty is not you or I.
> The sad thing is: any experienced automation engineer knows that depending on a human in the loop to behave rationally in an emergency is rank stupidity. And any death caused by engineering stupidity is a death we must avoid.
> This scenario smacks of the Armagh rail disaster (1889), the one that heralded the first regulation of railway networks - where the regulators stopped making suggestions and started throwing people in jail for non-compliance. 80 people had to die (a third of them children) to make that happen. I sincerely hope that we don't have to watch any children die before US regulators force Musk to put some adults in his development shop.
>  
> "... Thou marvel’st at my words: but hold thee still.
> Things bad begun make strong themselves by ill. "
>  
> Les
>  
>  
> -------------------------------------------------
> Les Chambers
> Director
> Chambers & Associates Pty Ltd
> www.chambers.com.au
> Blog: www.systemsengineeringblog.com
> Twitter: @ChambersLes
> M: 0412 648 992
> Intl M: +61 412 648 992
> Ph: +61 7 3870 4199
> Fax: +61 7 3870 4220
> les at chambers.com.au
> -------------------------------------------------
>  
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160714/7fe08a34/attachment-0001.html>