[SystemSafety] Difference between software reliability and astrology

[M Ellims]

Peter,

are you telling me that during some quite long meetings with FAA FADEC
specialists they were telling me porky-pies? I when over this specifically
because before the software is actually built, and tested over time, knowing
what the actual failure rate will be is more or less impossible with any
degree of accuracy.

Note I specifically asked the question in the meetings because of that issue
and the fact that the published guidance was not clear on the matter.

Note - that isn't an argument against doing it in an exploratory way, I
often enter 10E-6 in base events for software to see what happens and where
there may be problems but the actual evaluation is done with SW set to zero.

For reference Spitzer, "Digital Avionics Systems Principles and Practice",
2nd Ed. Says more or less the same thing...

	"that is, for the purpose of the FTA, the control laws are
considered perfect".

This is in relation to a basic i.e. undeveloped event E1 in figure 9.3
labelled as follows:

	"Control laws/or system logic deficient for environmental conditions
encountered"

Derek,

the assumption applies ONLY to the FTA i.e. the estimate of the probability
of failure against the target hull loss rate (hull == aircraft) of 10E-9.

The FAA knows full well that software isn't perfect but concedes that the
actual failure rate is not knowable so the FTA is used to ensure that the
hardware meets the requirements but assumes that the process required for
software development i.e. as laid out in DO178 is adequate to contain the SW
failure rates within an acceptable bound. However there is no numerical
targets for software reliability.

That being said, there is an ancient paper on avionics software reliability
I have laying around somewhere that suggests at DIL A something on the order
of 10E-8 seems achieve. John McD also did an estimate for automotive which
likewise seemed to 10E-8 was available relative to dangerous failures (but
it was based on some work i did so it's probably a bit dodgy :-) and I know
there are SW errors in automotive systems!

So in summary failures in software are considered separately from failures
in hardware.

The same approach is taken in 26262 but laid out explicitly in that 26262
requires an evaluation of failures rates for hardware (part 5) but not for
software... it seems to work reasonably well if you can get the requirements
right in the first place and do testing with the aim of actually trying to
cause a system to fail.

Not looking at anyone of course.



-----Original Message-----
From: systemsafety
[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
Derek M Jones
Sent: 22 August 2024 11:12
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Difference between software reliability and
astrology

Mike,

> Guidance from the FAA is the software included in any FTA analysis should
be assigned a failure rate of zero. The rational being that software failure
rates are in general cannot be reliably estimated and thus the
dependence/reliance on DO178.

Does this mean that it's not possible for anyone to report software
as the cause of a particular failure?

After all, if the failure rate is specified as zero, software can
never be considered a cause of failure.

A software failure rate of zero becomes a self fulfilling prophesy.

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
Manage your subscription:
https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety