[SystemSafety] Practical Statistical Evaluation of Critical Software
Peter Bernard Ladkin
ladkin at rvs.uni-bielefeld.de
Mon Mar 2 08:44:22 CET 2015
Derek,
to my mind, your contentions show that you don't really understand the difference between
abstraction (or description) and modelling. I wrote about this some twenty years ago. It keeps
coming up, indeed the last time was four weeks ago in Bristol.
People who build models try to make something which "looks like" the world as they conceive it.
Some bits fit, some bits may not - usually there are some such bits. They may do better or they may
do worse.
But how do they tell if their models are any good? Or where they do well or worse? The answer is
that models are compared with "reality". Now that's interesting - how does one "compare" with "reality"?
That is exactly the problem which John Locke had with his theory of perception, and of which
Berkeley offered what is still taken to be the definitive criticism. Refined in the meantime by
people such as George Moore and John Austin.
To "compare" with "reality", you can't build yet another model, for the same question arises and
remains thereby unanswered. "Comparing with reality" is an operation which is not modelling, but
which those who claim all-is-modelling do not describe.
If they try to do so, they invariably encounter description or abstraction. Abstraction is a matter
of selecting out common features between the "reality" you are attempting to grasp and some
construct you can manipulate, such that the manipulations of the construct correspond to actions in
the "reality" in some understandable way. (This is an attempt to put in one sentence something that
takes rather longer to set out carefully. I know it sounds circular in this sentence.)
I prefer to call it "description", because people generally know what a description is and what its
characteristics are, and that the point of a description is to be true of whatever one is
describing. That is, roughly speaking, the problem which Locke had - he couldn't account for
describing. When you correctly describe some object as "just blue, and not another color", you can
infer that object is not "black and not another color" and your inference is guaranteed to be true:
(all-)blue and (all-)black are contraries.
So if you have a pile of oranges and apples on the table, and you want to divide them as equally as
possible amongst the five kids at the birthday party, then arithmetic and arithmetic operations
might occur to you as a useful abstraction - as a useful description for what you want to do.
Abstraction is different from modelling, at least in that, when you draw a conclusion from
calculating with an abstraction, that conclusion is true (providing the abstraction is valid).
Whereas when you are working with a model, you don't generally know whether it is or not. You just
hope it is, just like most of the rest of your model.
So, when you draw conclusions from your arithmetic about how the apples and oranges are to be
divided most equally amongst the five children, you know that that is the correct answer - it is
guaranteed to be true (provided you did your arithmetic correctly). So you can just go ahead and do it.
If there is a car accident outside in the road - somebody braked on the curve, went off the road and
hit a wall, and you want to try to estimate how fast the car was going, then arithmetic is not going
to help you much. It's going to "apply" just as much as it does to apples and oranges, but it's not
going to get you anywhere near an answer to your question. Newtonian mechanics is going to do a lot
better. Ah, someone says, but Newtonian mechanics is a *model* - it makes assertions that we know
are *not true*. Well, I would reply, in this case not - it's really being used as an abstraction.
Yes, there are errors in Newtonian calculations which relativistic mathematics can highlight, but
those errors are bounded, and those error bounds can be propagated mathematically through the
calculations, enabling you to state your conclusion, which is your Newtonian-calculated figure
within bounds which your error calculation says pertain. That conclusion is as much guaranteed-true
as in the apples and oranges case with arithmetic.
Despite this issue being, essentially, over three hundred years old, and successfully (partly)
understood then, it recurs nowadays. Brian Cantwell Smith wrote an essay in the 1990's in which he
claimed you can't prove software correct, in which he made the Lockean veil-of-perception mistake.
James Fetzer did something similar. He talked about software being "correct" or not, without saying
correct with respect to what, and drew political conclusions which were politically unacceptable to
many computer scientists. Nobody pointed out in print the fallacy in his argument. He claimed (I
paraphrase) you couldn't mathematically guarantee anything to be true about real-world things.
That's nonsense. Here's my fridge. I'm going to prove to you that my fridge is correct. In order to
do that, I've got to try to tell you what it is for my fridge to be correct (or not). So here goes:
to be correct, my fridge is to fulfil the condition P(x) OR NOT-P(x). (Take P to be any decidable
property of my fridge.) It is then a trivial exercise to *prove mathematically* the correctness of
my fridge.
(The point of that exercise, obviously, is not to show anything useful about my fridge. It is to
show that, whatever his reasoning might be, Fetzer's conclusion is mistaken.)
Nevertheless, despite regular attempts to highlight the essential difference between description (or
abstraction) and modelling, people still don't get it.
For example, Nancy Leveson. It came up again in the question session after my talk in Bristol. I had
briefly indicated how Causal Fault Analysis can be used along with a judgement of the values of
quasi-Boolean variables in order to get a pruned - and I would claim helpful - decision graph about
the risks where there is much uncertainty (which applies to security risks in particular). Simon
Whiteley asked me if I had considering using STAMP, and if not why not. I said that STAMP is a
model, and CFA is a description technique, and (truth be told, I don't know whether I actually
stated this next - maybe Simon or someone else remembers) my aim is generally to describe causal
situations so that conclusions were guaranteed to be true given the truth of necessary assumptions,
whereas if one uses a model such as STAMP, one has to "buy into" the model (in particular the
"feedback model" of causality, which in contrast with the counterfactual interpretation used in WBA
and CFA has not yet withstood four decades of being hacked on by some of the smartest logical
minds), and thereby are the conclusions not guaranteed to represent "the world" - as one says, they
are only as good as the model might be. Nancy didn't like that. She is of the camp in which
everything is "modelling". I deferred a discussion on the issue of modelling versus description (and
if I hadn't, I'm sure Mike would have :-) ). I think if people can't acknowledge the difference over
twenty years, they are not going to do so in five minutes of repartee after a talk on something else.
She asked: how can I deal with human factors? (With WBA or CFA. Actually, she didn't ask, she simply
claimed I couldn't. I'm pretending she phrased it as a question.) Answer: like WBA/CFA deals with
anything else - put some in if you need them. (As Karsten Loer did in his WBA Diploma thesis from
seventeen years ago, which is Section 2 of the Causal Systems Analysis book which has been available
on the WWW for fourteen years.) Nancy maintains, however (at least she did afterwards in Bristol)
that I "can't deal" with human factors.
I think it would help system engineering a lot to get clear on the difference between modelling and
description.
I also think this is what Matthew was getting at in his note. The Urn Model has that unfortunate
word "Model" in its name, but in fact it is easy to extract the descriptive elements from the story:
"balls" don't need to be balls; they don't need to be "painted white (or black)"; there just need to
be selectable entities of two sorts, and exactly the same entities must be available for selection
in exactly the same way, repeatedly, and the selection action must satisfy certain conditions.
Certain kinds of software fulfils that description, just as the apples and oranges on the table
fulfil the arithmetic description. And for the software which fulfils that description, the
conclusions about time to first failure and MTTF and so on, derived from the mathematics, are valid.
You want to say "not good enough for me". OK. Maybe you want to use Erlang distributions or whatever
they are called. Fine. Go ahead. Nothing wrong with that if you can do it. If you can show how those
distributions describe the execution of software in some fundamental way, and the math gives you
useful conclusions.
But there is nothing in that observation at all which justifies any criticism of using the Urn Model
where it can be used, just as there is nothing in the insufficiency of arithmetic to describe the
dynamics of an accident to stop us using it successfully in dividing up apples and oranges equally
amongst five kids.
PBL
Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Je suis Charlie
Tel+msg +49 (0)521 880 7319 www.rvs.uni-bielefeld.de
More information about the systemsafety
mailing list