[SystemSafety] Difference between software reliability and astrology

[M Ellims]

Derik,

>> But what do the vendors do with regard to software failures?

Aerospace has a fairly robust incident reporting system, where the FAA defines an incident as "an occurrence other than an accident, associated with the operation of an aircraft, which affects or could affect the safety of operations".

Incidents have to be reported i.e. " The operator of an aircraft must immediately, and by the most expeditious means available, notify the nearest National Transportation Safety Board (NTSB) Field Office when..." with a defined list of events that have to be investigated including "Flight control system malfunction or failure."

Shooman 1996, Avionics software problem occurrence rates - looked at two FAA databases airworthiness directives (ADs) and service difficulty reports (SDRs) for instances where software errors were reported and decided from the abstract that...

"The average occurrence rate for the 6 resulting data sets was 0.15 per million operating hours. The non-occurrence of ADs for the remaining avionics was "bounded on the average"; yielding less than 0.02 occurrences/per million hrs."

Somewhere in the bowls of the filing cabinets is the original paper but I can't find it :-(


-----Original Message-----
From: Derek M Jones [mailto:derek at knosof.co.uk] 
Sent: 22 August 2024 13:57
To: M Ellims; systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Difference between software reliability and astrology

All,

> The FAA knows full well that software isn't perfect but concedes that the
> actual failure rate is not knowable so the FTA is used to ensure that the
> hardware meets the requirements but assumes that the process required for
> software development i.e. as laid out in DO178 is adequate to contain the SW
> failure rates within an acceptable bound. However there is no numerical
> targets for software reliability.
...
> So in summary failures in software are considered separately from failures
> in hardware.

Mike make some interesting interesting points about the FAA.
But what do the vendors do with regard to software failures?

When a failure occurs, do the people at the sharp end have a box
to tick specifying that this was/might be a software failure?
Unless raw data is being collected, software failure rate will
appear to be zero.  Even worse, the information needed to estimate
a rate will not be available.

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com