[SystemSafety] Difference between software reliability and astrology

[Derek M Jones]

Martyn,

> As a software engineer, I'm interested in software development and assurance methods that are based on science. The 
> great value of theory papers about software reliability is that they show the limits to the claims that should be mades 
> based on testing alone, and the limited (though valuable) stronger claim that could legitimately be made based on 
> testing supplemented by formal verification. These theory papers also reveal the assumptions that underpin such claims 
> and the restrictions that have to be placed on in-service updates, for example.

It's possible to concoct a wide variety of theories, based
on whatever assumptions one cares to chose.  The predictions
made by these theories may or may not match real world behavior.

> A great value of empirical evidence is that it can *disprove* scientific theories – assuming one is skilled enough to

As you say, without data there is no way of of comparing these
theories against real world behavior.

There is one theory paper that does not require data, because
it estimates worst case behavior based on zero fault experiences
"Worst case reliability prediction based on a prior estimate of
residual defects." by Bishop and Bloomfield.

> I see from your astrology reference that you consider their experiment to have generated "evidence". Please say what 
> confidence you have in each of their results and on what basis (mentioning, for example, their sample sizes and their 
> selection methods).

What I said was:
"Data is available to check the predictions of astrologists"

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com