[SystemSafety] 1. View of Computer Reliability in the Law (English and US) and 2. Argumentation

[Phil Koopman]

On 7/2/2021 5:53 PM, Les Chambers wrote:
> What does she mean by ‘needed to be closely examined … adversarial examination of code’ ?
> Do you mean rerun the entire V&V program Susan? That is assuming you could find the correct
> versions of all the design documents and (choke) a complete, correct and unambiguous statement
> of software requirements. Who has that kind of money?

Les,

I understand your concern about the practicality of independent V&V of 
software like this, but that's not quite the point that is being made.

My understanding of the argument is that if the notional "accuser" is a 
piece of software, someone accused should have the opportunity to 
"question" whether the "accuser" is making credible claims. That 
"questioning" might be expensive, but they should nonetheless have the 
opportunity if they have resources to pursue it.

An adversarial examination need not re-run the entire V&V program, since 
an adversary would not need to prove the code is fit for purpose. 
Rather, a defendant would seek evidence that the code is NOT fit for 
purpose.  One big defect found (failure to do what the prosecution says 
it does), a systemic lack of quality, lack of an acceptable V&V paper 
trail, defective configuration management, or other deficiencies in 
development and application of the software might suffice to establish 
reasonable doubt, especially for criminal cases.

As to money, that is what the US Class Action system and other 
collective litigation approaches are for.  If you have enough 
high-stakes cases on the table and/or a deep-pockets benefactor 
foundation, the pooled resources can indeed take on analysis of a large 
complex piece of code with enough potential for success to make it worth 
doing.

I'm not saying the legal system is perfect, but if a judge were to 
permit examining source code, over the long term it could well make a 
practical difference.

(BTW I'm not a lawyer and not giving legal advice.)

-- Phil

-- 
Prof. Phil Koopman   koopman at cmu.edu
(he/him)             https://users.ece.cmu.edu/~koopman/