[SystemSafety] a public beta phase ???

[Peter Bernard Ladkin]

On 2016-07-17 19:45 , Martyn Thomas wrote:
> One of us is missing the point. I'll assume it's you :-)

I'm happy to agree I'm missing the point.

Tesla is putting a car on the road with automated-driving capability that is "public beta". That
presumably means they are interested in incremental improvement after anomalies. That means, I hope,
causal analysis of incidents and implementing "lessons learned".

No matter what one's idea of how to deploy safety-critical systems in new applications, that is
going to happen anyway, I suggested.

Is this enough? No. And I said so. There should be conditions, and assurances on those conditions,
on initial deployment.

You suggested that a condition should be that no more accidents result from the deployment of the
technology than were apparent before. I said that that was MGS/GAMAB and suggested that it was
neither necessary nor sufficient.

One way of deploying automated safety technology is to do so in a targeted manner. Not: we have a
cool autopilot; but: we have implemented a function which will eliminate a class of accidents.

Eliminating a class of accidents does not necessarily mean that no new accidents occur. If you
inhibit speed to within posted limits, then you are restricting available performance. Suppose you
overtake a slow moving vehicle, with plenty of space before oncoming traffic. And that driver
doesn't like it and speeds up as you overtake. You could, before the inhibitor, give even more gas
and complete the manoeuvre as planned. Now, your inhibitor prevents you from doing that, enabling a
more limited set of avoidance manoeuvres. Maybe more accidents will occur because of that. But they
obviously won't occur because of overspeed. Maybe people will play a new game: driving slow to
encourage overtaking and then speeding up when being overtaken to induce conniptions in the
overtaking vehicle which is speed limited. The fact that this is a plausible story about a new
situation should suffice to say that it cannot be proven that even a speed-limiter will result in
fewer accidents. But people will adapt; the games will stop; there will be no more cars travelling
over the posted limits and it is thereby almost certain that the number of accidents will be
eventually reduced.

So I think your condition is too strong.

PBL

Prof. Peter Bernard Ladkin, Bielefeld, Germany
MoreInCommon
Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160717/2ba6704b/attachment.pgp>