[SystemSafety] a public beta phase ???

[Martyn Thomas]

Peter

You seem to be arguing that "test and fix" works. Let me offer the IBM
PLI-F compiler as a counterexample. It was complex and buggy. Bugs were
found and fixed but on average each fix introduced more bugs. Finally
the compiler had to be withdrawn.

If your development team introduces an error in every 100 LoC they write
(putting them in the best 1%) then when your average fix size exceeds
100 LoC things start looking bad.

I recognise that most bugs in a driverless car won't cause an accident.
But I still don't like test-and-fix as a strategy.

Martyn



On 17/07/2016 20:45, Peter Bernard Ladkin wrote:
>
> On 2016-07-17 19:45 , Martyn Thomas wrote:
>> One of us is missing the point. I'll assume it's you :-)
> I'm happy to agree I'm missing the point.
>
> Tesla is putting a car on the road with automated-driving capability that is "public beta". That
> presumably means they are interested in incremental improvement after anomalies. That means, I hope,
> causal analysis of incidents and implementing "lessons learned".
>
> No matter what one's idea of how to deploy safety-critical systems in new applications, that is
> going to happen anyway, I suggested.
>
> Is this enough? No. And I said so. There should be conditions, and assurances on those conditions,
> on initial deployment.
>
> You suggested that a condition should be that no more accidents result from the deployment of the
> technology than were apparent before. I said that that was MGS/GAMAB and suggested that it was
> neither necessary nor sufficient.
>
> One way of deploying automated safety technology is to do so in a targeted manner. Not: we have a
> cool autopilot; but: we have implemented a function which will eliminate a class of accidents.
>
> Eliminating a class of accidents does not necessarily mean that no new accidents occur. If you
> inhibit speed to within posted limits, then you are restricting available performance. Suppose you
> overtake a slow moving vehicle, with plenty of space before oncoming traffic. And that driver
> doesn't like it and speeds up as you overtake. You could, before the inhibitor, give even more gas
> and complete the manoeuvre as planned. Now, your inhibitor prevents you from doing that, enabling a
> more limited set of avoidance manoeuvres. Maybe more accidents will occur because of that. But they
> obviously won't occur because of overspeed. Maybe people will play a new game: driving slow to
> encourage overtaking and then speeding up when being overtaken to induce conniptions in the
> overtaking vehicle which is speed limited. The fact that this is a plausible story about a new
> situation should suffice to say that it cannot be proven that even a speed-limiter will result in
> fewer accidents. But people will adapt; the games will stop; there will be no more cars travelling
> over the posted limits and it is thereby almost certain that the number of accidents will be
> eventually reduced.
>
> So I think your condition is too strong.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Bielefeld, Germany
> MoreInCommon
> Je suis Charlie
> Tel+msg +49 (0)521 880 7319  www.rvs-bi.de
>
>
>
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160718/4962a670/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 560 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160718/4962a670/attachment-0001.pgp>