[SystemSafety] a discursion stimulated by recent discussions of alleged safety-critical software faults in automobile software

[Nancy Leveson]

I'd like to suggest that mixing up engineering and law is a mistake. There
are, or at least should be, different goals. When I am involved in the
investigation of major accidents, I often find that the lawyers prevailed
over the engineers in making engineering decisions. In other cases, I have
gone in and told the engineers what they need to do to make their systems
safe. They agree but tell me that their lawyers will not let them do what I
am suggesting. I wash my hands of the company then and try to make sure
that I sell any stock I own in them.

In addition, the legal definition of "cause" is not necessarily the same as
the engineering definition of "cause." Nor need it be the same as, again,
the goals are different. Engineers are not trying to find one person or
organization with deep pockets or to put someone in jail.

Nancy


On Tue, Nov 12, 2013 at 1:52 AM, Peter Bernard Ladkin <
ladkin at rvs.uni-bielefeld.de> wrote:

> It seems worth while making again a point I have made before.
>
> It is not about blame. Which, by the way, I wouldn't necessarily call an
> emotion (Wikipedia, for example, thinks it is an act). It is about
> assignment of responsibility for a deleterious event with a view to
> dispensing compensation. This is a general principle of human behavior and
> lawmaking for thousands of years and occurs in many if not all human
> societies. I won't argue here the case for compensating people for harm you
> have caused them. I'm glad we adhere to it and that I don't live 1600 years
> ago.
>
> So, if you are a 1970's hotel owner and a rock group trashes some of your
> rooms, you are entitled to a determination of responsibility, and adequate
> compensation from those deemed responsible. Since that will often be
> disputed (likely not by a 1970's rock group, for which it was a source of
> pride), it needs to be decided by the appropriate means, which for us is a
> court of law.
>
> It used to be the case in GB that hordes of foreigners came ashore from
> boats, took what they wanted, trashed the restaurants as if they were
> Bullingdon boys, and took women into slavery. They had to be fought off.
> When this started being successful, they quit (apart from those who stayed,
> which ruined their business model another way). Every three-year old who
> has played in a sandbox knows this phenomenon, which manifestly does not
> stop when one is older: John Kenneth Galbraith wrote about the power of
> large corporations and the consequences for human society between 40 and 55
> years ago. So there is also another  point to this kind of action:
> resistance stops other people doing stuff.
>
> Toyota knew they had spaghetti code in this acceleration-control kit. They
> wrote so themselves, which you can see in the evidence. They also knew and
> know the consequences of such complexity, namely a lack of control over the
> behavioral properties of the program. That is also in the evidence. It
> didn't stop them using the code again and again (it was still in the 2010
> model year, apparently). That won't continue. For example, they have
> recently signed a contract with Altran UK to develop examples of useful
> code which is free of run-time error.
>
> If you don't like principles of fairness and responsibility, and developed
> organisations (the courts) with the power to set those principles of
> fairness for everyone and every organisation without exception, just try
> doing without it..........
>
> PBL
>
> Prof. Peter Bernard Ladkin, University of Bielefeld and Causalis Limited
>
> On 12 Nov 2013, at 03:07, "Les Chambers" <les at chambers.com.au> wrote:
>
>  What bothers me is the alarming repeat performances we have of these
> disasters. And the eye-watering sums of money spent on forensics and
> retribution. These events are typically passed over to the legal profession
> who proceed to dine out on the assignation of blame.......
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>


-- 
Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142

Telephone: 617-258-0505
Email: leveson at mit.edu
URL: http://sunnyday.mit.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20131112/50b3ef07/attachment.html>