[SystemSafety] Difference between software reliability and astrology

[Prof. Dr. Peter Bernard Ladkin]

On 2024-08-22 01:59 , Phil Koopman wrote:
> So my takeaway is that 1e-9 applies to all aircraft OF ONE TYPE and not all aircraft in the fleet. 
> Keep in mind this was written in 1988 when the skies were a lot less crowded.  So someone did some 
> back of envelope math on flight hours per day, number of aircraft of a popular type, and airframe 
> lifetime and came up with this number.
>
> Also note that this is for a "failure condition" and is not the acceptable failure level for the 
> aircraft. I believe there is an assumption that perhaps 10 different failure conditions might all 
> be possible, making the aircraft loss rate an order of magnitude worse per hour (but I might not 
> be remembering the number 10 correctly -- I don't know if it is really written down anywhere).

It is the product of about three decades of post-WWII experience and is written down in Chapter 4 of 
Systematic Safety, E. Lloyd and W. Tye, Civil Aviation Authority London, 1982, which talks about 
JAR-25 rather than FAA AC 25.1309.

[begin quote p37]

.... the broad intention is that system failures resulting in Catastrophic Effects should virtually 
never occur in the whole fleet life. If one assumes that as many as 100 individual Catastrophic 
Effects emerged from the safety assessment of all the systems in an aircraft types, the total 
maximum risk implied by the requirements would be 100 x 10^(-9) = 10^(-7). This would mean that with 
a fleet of 100 aircraft of a type, each flying 3,000 hours per annum, one or other of the various 
Catastrophic Effects might be expected to turn up once in 30 odd years, which is close to the 
concept of "virtually never".

[PBL Note: parentheses added to exponents, were not in original]

[end quote]

Notice they are only talking about single failure events with Catastrophic Effects. They are well 
aware that most accidents result from a number of different factors. But some didn't - the Comet 
break-ups, for example. Or the problems with wings coming off Wellingtons (I think it was) in WWII, 
which led to the emergence of and importance of metallurgical engineering in aerospace. Post war, 
there was a lot of effort put in to counter such single failures, to great success. Wings don't come 
off commercial aircraft nowadays.

Note also that the fleet size is very different from what it is nowadays. Flight International puts 
out a directory of in-service commercial aircraft types once a year. I don't subscribe any more, but 
Wikipedia helps. Boeing 747: 434 in service. Boeing 787: 1115 in service.

The two "biggies" are: Boeing 737: between 11,000 and 12,000 delivered and 18,000+ ordered. The 
Airbus A320 is similar. (Note: not all those delivered will still be in service.)

Lloyd and Tye is many years out of print, unfortunately.

The leading expert on this history and its rationale is John Downer, of Uni Bristol. He has just 
published Rational Accidents (MIT Press, 2024). John is on this list.

PBL

Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
www.rvs-bi.de