[SystemSafety] 1. View of Computer Reliability in the Law (English and US) and 2. Argumentation

[Les Chambers]

Interesting 
 but

There seems to be a disconnect between the legal profession / computer science community and 
practitioners who build complex systems. It’s sad that the voice of the practitioner is seldom heard.

I refer to Susan Landau’s article Dangers Posed by Evidentiary Software—and What to Do About 
It
See:
‘TrueAllele has not undergone this type of close scrutiny. Even if the science behind it had, the 
software implementing it needed to be closely examined as well. Emphasizing the points we had 
made regarding the need for adversarial examination of code, the court ruled, "Hiding the source 
code is not the answer. The solution is producing it under a protective order.’

What does she mean by ‘needed to be closely examined 
 adversarial examination of code’ ? 
Do you mean rerun the entire V&V program Susan? That is assuming you could find the correct 
versions of all the design documents and (choke) a complete, correct and unambiguous statement 
of software requirements. Who has that kind of money?

I thought the profession had given up on proving that anything but a ten line program produces 
correct output for all combinations of inputs in all environments. Instead we have retreated to the 
second line of defence where we give developers the benefit of the doubt if they can prove they 
used certain development processes.

After 46 years of writing and testing software I’d give it no credibility at all as a single source of 
evidence. It concerns me that the legal profession is so far behind the front of a rapidly moving 
technology where software, redolent  with artificial intelligence cannot even be trusted to comply 
with its original software requirements as it learns and changes its behaviour as a function of its 
environment. 

Even voice can be edited with the ease of a word processor. Video evidence can be deep faked.

I look forward to the return of Constable Plod, “What’s this then?”

Les


> Susan Landau has published an article in the Lawfare blog about attitudes of English and US 
courts 
> towards the evidence generated by, or about the behaviour of, computers in cases they 
consider.
> 
> https://www.lawfareblog.com/dangers-posed-evidentiary-software—and-what-do-about-it
> 
> She refers to Paul Marshall's masterful account to the University of Law of the PO Horizon affair 
> https://www.postofficetrial.com/2021/06/marshall-spells-it-out-speech-to.html
> 
> Susan also singles out Paul's observation that ".... writing on a bit of paper in evidence is only 
> marks on a piece of paper until first, someone explains what it means and, second, if it is a 
> statement of fact, someone proves the truth of that fact."  This applies strongly also to 
arguments 
> presented in assurance cases for safety-critical systems.
> 
> Rather than say more here, I refer to my blog post 
> https://abnormaldistribution.org/index.php/2021/06/27/computer-reliability-in-legal-arguments-
with-some-observations-about-arguments/ 
> 
> PBL
> 
> Prof. Peter Bernard Ladkin, Bielefeld, Germany
> ClaireTheWhiteRabbit RIP
> Tel+msg +49 (0)521 880 7319  www.rvs-bi.de



--
Les Chambers
les at chambers.com.au
+61 (0)412 648 992