[SystemSafety] Elephants, dinosaurs and integrating the VLA model
Peter Bernard Ladkin
ladkin at causalis.com
Thu Aug 3 14:19:56 CEST 2023
On 2023-08-03 13:48 , Les Chambers wrote:
> Peter
> Your comment:
> "Martyn already observed on 2023-06-27 that there are legal requirements which
> constrain deployment of safety-related systems. That legal requirement in the
> UK and Australia is 77 years old. Your question seems to be suggesting that you
> somehow think it, and other constraints, might no longer apply. Well, they do.
> As Martyn said "AI doesn't change that.
> In the UK or Australia, developer and deployer must reduce risks ALARP.”
>
> … is righteous … that is, if decreed by a king ruling by fiat (Latin for “let
> it be done”).
Yes, well, by far the largest part of Causalis Limited's clients has been lawyers working on
accidents. You may think paying close attention to applicable law is "righteous". I think of it as
common sense.
> The interesting question is, how does one PROVE (to an auditor or a judge) that
> one has reduced risks ALARP if one’s delivered system’s behaviour is initiated
> from a neural network?
Yes, it is an interesting question. The current answer is that you show the neural network has
limited scope for action and present the hazard analysis that shows that that limited scope leads
only to acceptable risk, and that you can't easily do any better.
And if you can't do that, you're screwed (to coin a term).
> Case study: Elaine Herzberg, killed by a self-driving Uber in Tempe, Arizona in
> 2018. The system did not classify her as a pedestrian because she was crossing
> without a crosswalk; the neural net did not include consideration for
> jaywalking pedestrians.
What is the outcome of the case study?
I can't see what you are getting at with this line of thought.
PBL
Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
Tel+msg +49 (0)521 880 7319 www.rvs-bi.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20230803/0517492a/attachment.sig>
More information about the systemsafety
mailing list