<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I guess that is the nub of the argument with regard to systematic
faults.<br>
<br>
- as a standalone artefact, the software is "faulty" rather than
reliable</p>
<p>As a component in some operating environment, E, "reliability"
becomes meaningful.</p>
<p>i.e. the probability of failure of the software component is</p>
<p> Pr_E( not A)</p>
<p>i.e. <b>In environment E</b> , the probability of an input value
that is (not A).</p>
<p>Now suppose we can specify which input values in a program are
correct or faulty (i.e. we know A and not A precisely).</p>
<p>could you tell me exactly when the program will fail in operation
?</p>
<p>- Obviously (apart from the special case where it always fails,
Pr_E( not A)=1), <br>
you cannot do this unless you have a crystal ball that predicts
future inputs in environment E with total precision.</p>
<p>But you can express the future occurrence of A in probabilistic
terms, i.e. how often (not A) will arise in environment E.</p>
<p>In practice of course you do not actually have information about
(not A)<br>
- these are the bugs you have not found yet.<br>
</p>
<p>But testing in operational environment E allows you to set a
confidence bound on Pr_E(not A)<br>
- even if you do not know where the bugs are located in the
program input space.</p>
<p>And that is what statistical testing does,<br>
but as others have already said, the reliability bound is
conditional on the operating environment E.</p>
<p>Peter</p>
<p>PS<br>
</p>
<p>There has been some work on the worst case change in the bound
that can be claimed for a different operational profile E'<br>
<a class="moz-txt-link-freetext" href="https://www.researchgate.net/publication/307555914_Deriving_a_frequentist_conservative_confidence_bound_for_probability_of_failure_per_demand_for_systems_with_different_operational_and_test_profiles">https://www.researchgate.net/publication/307555914_Deriving_a_frequentist_conservative_confidence_bound_for_probability_of_failure_per_demand_for_systems_with_different_operational_and_test_profiles</a></p>
<p>where idea of extra "padding tests" is introduced to maintain the
claimed bound for specified departures from E.<br>
</p>
<p>There is also an alternative strategy - "fair" testing - where
testing is based on the likelihood of a particular input A being
faulty <br>
(based on code path length rather than E), which can reduce
sensitivity of reliability bounds to profile change.</p>
<p>
<a class="moz-txt-link-freetext" href="https://www.researchgate.net/publication/234802669_Rescaling_reliability_bounds_for_a_new_operational_profile">https://www.researchgate.net/publication/234802669_Rescaling_reliability_bounds_for_a_new_operational_profile</a><br>
</p>
<div class="moz-cite-prefix">On 16/09/2020 01:02,
<a class="moz-txt-link-abbreviated" href="mailto:hugues.bonnin@free.fr">hugues.bonnin@free.fr</a> wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1466672325.226775889.1600214564205.JavaMail.root@zimbra60-e10.priv.proxad.net">
<pre class="moz-quote-pre" wrap="">Hi all,
I have an alternative "toy" to propose: do you think that this software is reliable (written in ada-like code)?
begin
if A then
do_nothing
else
fail --potentially hurt and kill people
end if
end
The specification of the software is to do nothing;
NB: I'm not asking if it is the best implementation, whatever the criteria are, but just : "is it reliable?"
regards,
Hugues
----- Mail original -----
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">De: "Peter Bernard Ladkin" <a class="moz-txt-link-rfc2396E" href="mailto:ladkin@causalis.com"><ladkin@causalis.com></a>
À: <a class="moz-txt-link-abbreviated" href="mailto:systemsafety@lists.techfak.uni-bielefeld.de">systemsafety@lists.techfak.uni-bielefeld.de</a>
Envoyé: Mardi 15 Septembre 2020 19:58:45
Objet: Re: [SystemSafety] What do we know about software reliability?
Bev and I and Dewi have a colleague who poses the following question.
"We have clients who have installed hundreds of [examples of our kit]
over the last ten years, and
have never seen any failure. They want to use it in further systems
that they build. What arguments
do we/they need to provide in order validly to justify such further
use?"
So, what is the answer to that question?
PBL
Prof. Peter Bernard Ladkin, Bielefeld, Germany
Styelfy Bleibgsnd
Tel+msg +49 (0)521 880 7319 <a class="moz-txt-link-abbreviated" href="http://www.rvs-bi.de">www.rvs-bi.de</a>
_______________________________________________
The System Safety Mailing List
<a class="moz-txt-link-abbreviated" href="mailto:systemsafety@TechFak.Uni-Bielefeld.DE">systemsafety@TechFak.Uni-Bielefeld.DE</a>
Manage your subscription:
<a class="moz-txt-link-freetext" href="https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety">https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety</a>
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">_______________________________________________
The System Safety Mailing List
<a class="moz-txt-link-abbreviated" href="mailto:systemsafety@TechFak.Uni-Bielefeld.DE">systemsafety@TechFak.Uni-Bielefeld.DE</a>
Manage your subscription: <a class="moz-txt-link-freetext" href="https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety">https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety</a></pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Peter Bishop
Chief Scientist
Adelard LLP
24 Waterside, 44-48 Wharf Road, London N1 7UX
Email: <a class="moz-txt-link-abbreviated" href="mailto:pgb@adelard.com">pgb@adelard.com</a>
Tel: +44-(0)20-7832 5850
Registered office: 5th Floor, Ashford Commercial Quarter, 1 Dover Place, Ashford, Kent TN23 1FB
Registered in England & Wales no. OC 304551. VAT no. 454 489808
This e-mail, and any attachments, is confidential and for the use of
the addressee only. If you are not the intended recipient, please
telephone 020 7832 5850. We do not accept legal responsibility for
this e-mail or any viruses.</pre>
</body>
</html>