[SystemSafety] Post Office Horizon System

[Peter Bishop]

See inline comments below

On 10/01/2021 14:48, Derek M Jones wrote:
> Peter,
>
>>>> The PO used some very dodgy statistical arguments to support their
>>>> reliability claim
>>>> - rebutted in:
>>>>
>>>> https://ials.blogs.sas.ac.uk/2019/06/26/the-use-of-statistics-and-software-code/
>>>>
>>>>
>>>
>>> I think that both sides got it wrong:
>>> http://shape-of-code.coding-guidelines.com/2021/01/07/likelihood-of-a-fault-experience-when-using-the-horizon-it-system/
>>>
>>>
>>>
>> In what way did the other side get wrong?
>
> They both assumed there was enough information available to calculate
> the likelihood of a fault being experienced.
>
>> Most of the material submitted identified fallacies in the PO
>> statistical arguments
>> (including the lottery analogy that you included the Shape of Code
>> document).
>
> This argument does not appear in the links you gave or the Ladkin et al
> papers.  Do you have a reference?

It is in the reference I gave

https://ials.blogs.sas.ac.uk/2019/06/26/the-use-of-statistics-and-software-code/


See abstract below:


      Discussion and conclusions

I find it amazing that Dr Worden’s seriously flawed analysis could be
viewed as credible evidence in a court of law.

Looking at the probability that an account submission can fail and
saying it is tiny is meaningless on its own. By analogy, it is illogical
to say that if there is only a 1 in a million chance of winning a
lottery, ergo any person who claims to have won the lottery must be
lying. This argument ignores the fact that increasing the number of
people who buy tickets will increase the probability that somebody will
win (even if your own chances remain the same). For example, if we know
that 10 million people buy tickets, we would not be at all surprised to
hear the 10 people won the lottery that week.

To perform a statistical analysis to determine whether the claimant’s
claims are credible, we should start from the hypothesis that all
branches are potential victims of random Horizon failures, then ask what
conditions are needed to produce 500 victims and then consider whether
these conditions are credible.

We showed that only 31 bugs similar to the Suspense Account bug are
needed to cause submission failure in 500 branches. This number of
residual bugs is entirely credible for a complex real-time system, and
in practice there could be many more than this (even in a mature
20-year-old system).

As a result of these analyses we consider that it is entirely credible
that issues experienced by the 500 claimants could have been caused by
flaws in the Horizon software.

-- 

Peter Bishop
Chief Scientist
Adelard LLP
24 Waterside, 44-48 Wharf Road, London N1 7UX

Email: pgb at adelard.com
Tel:  +44-(0)20-7832 5850

Registered office: 5th Floor, Ashford Commercial Quarter, 1 Dover Place, Ashford, Kent TN23 1FB
Registered in England & Wales no. OC 304551. VAT no. 454 489808

This e-mail, and any attachments, is confidential and for the use of
the addressee only. If you are not the intended recipient, please
telephone 020 7832 5850. We do not accept legal responsibility for
this e-mail or any viruses.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20210110/e0ac9ee9/attachment.html>