[SystemSafety] Bossavit's Leprechauns book

Tue Dec 11 15:45:38 CET 2018

Is this forum still active?  Haven’t seen any traffic since last Friday?

Brian Smith

From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de<mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de>> on behalf of Matthew Squair <mattsquair at gmail.com<mailto:mattsquair at gmail.com>>
Date: Friday, December 7, 2018 at 7:15 PM
To: Nick Tudor <njt at tudorassoc.com<mailto:njt at tudorassoc.com>>
Cc: "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: Re: [SystemSafety] Bossavit's Leprechauns book

Sounds like they were aiming for a randomized trial?

Matthew Squair

MIEAust, CPEng
Mob: +61 488770655
Email; Mattsquair at gmail.com<mailto:Mattsquair at gmail.com>
Web: http://criticaluncertainties.com

On 7 Dec 2018, at 6:47 pm, Nick Tudor <njt at tudorassoc.com<mailto:njt at tudorassoc.com>> wrote:

You’re right, conducting trials is not easy, unless you have partial funding to conduct such benchmarking trials. We were part of an industrial trial last year using one of our embryonic FM tools vs a baseline of people only vs an incumbent. The trial seeded 48 errors into a real exemplar; I understand that these were typical and atypical errors which were ‘designed’ by both a University and the industrial participants. In other words, we had nothing to do with designing the trial; we merely provided our embryonic tool into the trial. The 3 strands of the trial were conducted by 3 different people on 6 parts of an ASIL D software design so that there was no pollution of knowledge. One of the 6 parts was repeated in another company; ie completely independently. The experiment was design by a specialist metrics company and t’was they that policed and produced the results. The experiment was to measure the time taken to satisfy the review process between requirements and design in Simulink.

Not only did our tool find all the errors, we also found an extra one. Against the benchmark savings of 60-80% were found and against the incumbent 50-60%. The repeated part gave almost exactly the same results as the other; within a few %  for each of the 3 strands. There is a paper; if you’re interested contact me at njt at drisq.com<mailto:njt at drisq.com>. The embryonic tools are now almost at production level (Jan’19) and we will be talking about other uses at SSS’18.

On Thu, 6 Dec 2018 at 23:10, Olwen Morgan <olwen at phaedsys.com<mailto:olwen at phaedsys.com>> wrote:

While, ideally, we should base our engineering on what we can prove to
be true, the argument that things are not true because there is no hard
evidence for them is scientific dilettantism. There is ample reason to
question the validity of various claims in software engineering not
because they are not true but because there are many reasons why the
quality of the evidence may be sufficiently poor that they are not
proven. Not least among such reasons is the technical difficulty and
cost of conducting properly controlled trials.

Not true and not proven are not synonymous - as we all learned from the
proof of Fermat's Last Theorem by Andrew Wiles.

If you want to be a hard-nosed empiricist, you need to be rigorous about
the design of the experiments by which you test hypotheses. If you want
to question the validity of published results, that's fine ... but if
you do, you need to look at all the circumstances in which a study has
been carried out and document any flaws that you find in the methods
used. Claims of no proof must be supported by hard evidence of flawed
investigation.

How many reported studies does Bossavit examine with due scientific rigour?

Find that out and you'll know whether he's hoisted himself by his own
petard.

Olwen

On 06/12/2018 22:00, Les Chambers wrote:
> Derek
> Thanks but Shock horror.  Sounds like he is devaluing some of the fundamental truths of SE.
> For example the ‘rising cost’ concept is the basis for spending millions on V&V. Righteous
> dollars.
> Have we a tech Trump amongst us pedaling fake news with honeyed words?
> Seriously though, this type of BS does not advance the SE canon.
> Bin it!
> Les
>
>> Les,
>>
>>> Is Bossavit supporting or rejecting the legends. Having read his book, what does he want
> you to
>>> do with the information.
>>   From the open chapter:
>>
>> "The software profession has a problem, widely recognized but
>> which nobody seems willing to do much about. You can think of
>> this problem as a variant of the well known Â¿telephone gameÂ¿,
>> where some trivial rumor is repeated from one person to the next
>> until it has become distorted beyond recognition and blown up
>> out of all proportion.
>>
>> Unfortunately, the objects of this telephone game are generally
>> considered cornerstone truths of the discipline, to the point that
>> their acceptance now hinders further progress.
>>
>> It is not that these claims are outlandish in themselves; they
>> started as somewhat reasonable hypotheses. The problem is that
>> they have become entrenched as Â¿factÂ¿ supposedly supported
>> by Â¿researchÂ¿, and attained this elevated status in spite of being
>> merely anecdotal.
>>
>> How we got there
>>
>> One of the ways that anecdote persists is by dressing itself up
>> in the garments of proper scholarship. Suppose you come across
>> the following claim for the first time:..."
>>
>>> Les
>>>
>>>> Les,
>>>>
>>>>> PBL
>>>>> Please explain. Is Bossavit denigrating these concepts by calling them memes, myths or
>>> legends? Is he
>>>> Peter has jumbled together commentary on Bossavit's book, commentary
>>>> on what people on this list have been talking about and his own views.
>>>>
>>>> Bossavit writes in a very readable style and it is clear what
>>>> his discovered (or rather not discovered, i.e., data backing up
>>>> the claims investigated).
>>>>
>>>> --
>>>> Derek M. Jones           Software analysis
>>>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com<http://shape-of-code.coding-guidelines.com>
>>>> _______________________________________________
>>>> The System Safety Mailing List
>>>> systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
>>>> Manage your subscription: https://lists.techfak.uni-
> bielefeld.de/mailman/listinfo/systemsafety<http://bielefeld.de/mailman/listinfo/systemsafety>
>>>
>>>
>>> --
>>> Les Chambers
>>> les at chambers.com.au<mailto:les at chambers.com.au>
>>> +61 (0)412 648 992
>>>
>>>
>>>
>> --
>> Derek M. Jones           Software analysis
>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com<http://shape-of-code.coding-guidelines.com>
>
>
> --
> Les Chambers
> les at chambers.com.au<mailto:les at chambers.com.au>
> +61 (0)412 648 992
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
>
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
--
Nick Tudor
Tudor Associates Ltd
Mobile: +44(0)7412 074654
www.tudorassoc.com<http://www.tudorassoc.com>
[http://www.tudorassoc.com/wpimages/wpb4e71a5c_0f.jpg]

77 Barnards Green Road
Malvern
Worcestershire
WR14 3LR
Company No. 07642673
VAT No:116495996

www.aeronautique-associates.com<http://www.aeronautique-associates.com>
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20181211/cc0b807f/attachment.html>