[SystemSafety] The bomb again

[Nancy Leveson]

I don't know what prompted Peter's critique of STAMP. But I feel compelled
to respond. [Peter's comments are in italics]

*Two hundred fifty years ago, David Hume proposed two characterisations of
cause which have persisted ... Ten years ago, Nancy announced that she had
a new conception of causality, which was embodied in STAMP. I saw, and
continue to see, a problem in redefining a concept which has had a good and
productive run in science for three centuries (if not two and a half
millenia). It could well be that this new concept is a very useful concept;
it could be very helpful in identifying areas of interest in accident
investigation; indeed, judging by the interest in STAMP I imagine many
people think it is. But why not choose a different word for it?*
*
*
The world has changed a lot in the past 250 years, particularly the world
of engineering. Whole new fields have been created, some to explain the
behavior of new machines. An example is the development of thermodynamics
to explain the behavior of steam engines and their tendency to explode and
cause accidents. After WWII, the growth of new technology exploded, with
the notable introduction of the computer, which has revolutionized the
world of engineering. New types of math and engineering concepts were
invented (or resurrected, such as Boolean algebra) to keep up. To say that
a philosophical concept was good enough 250 years ago so it should still be
good enough today reminds me of Kelvin's [alleged] statement in 1900
that "There
is nothing new to be discovered in physics now; All that remains is more
and more precise measurement."

As we build more complex systems, new engineering techniques (and sometimes
new math) is needed to keep up. Accident causes are changing as the design
of our systems changing, or at least increasing, as the design of our
systems change. The argument in my new book is that what was adequate in
the past is no longer adequate. STAMP is a new accident causality model
that extends the older models to include new accident causes in our
increasingly complex world (it still includes the old ones, i.e., the old
models as a subset). In reality, STAMP is not really "new" in that it is
simply an application of systems theory to the specific concept of safety.
Actually, much of what is in STAMP is implied in the writings of the
aeronautical engineers who invented System Safety in the late 1940s and
1950s (such as C.O. Miller who claimed to have come up with the term System
Safety). Systems theory itself was a reaction to exactly the kind of
philosophical thinking that Peter's alludes to because the traditional
concepts did not fit the world of engineering that was beginning to appear
in the 1940s and 1950s. ISystems theory was a reaction to analytic
reductionism (by those like Descartes) which assumes that a system is the
sum of its parts. Systems theory, which underlies System Engineering (which
arose at the same time in the 1950s) instead assumes that a system can be
more than the sum of its parts. System Safety and System Engineering were
closely allied in 1950s missile defense systems for which they were
originally developed. Analytic reduction was an important concept the
development of modern physics, but after 200 years, engineers started to
build systems for which something new was needed.

*People who like STAMP could *obviously* use WBA for parts of what they do
- the two methods are compatible. And they would see the same advantage as
other WBA users. The only hindrance to such practice is use of the word
"causal" for two different concepts.*
*
*
*

The other thing about using STAMP is you have to buy the model. Now, I'm
sure it is helpful, because the people developing STAMP are very smart and
very dedicated and have been at it for a decade. But is the model right?
One might well be able to persuade engineers that the STAMP
social/organisational model is the bee's knees, but it is a quite different
matter to persuade the experts in those things, the organisational
scientists.

Constance Perrin wrote a book in which she investigated some incidents at
nuclear power stations and came to the conclusion that there was a tension
between the way the plant was conceived to work organisationally and the
architecture of plant operations impregnated in the minds of the operators,
who came mostly from the "nuclear navy", which had/has a modus operandi
completely different from the intended plant-operations architecture. A
crucial insight. It is not obvious to me how a STAMP analysis would lead
you to the same conclusion. (Maybe a good project to try?) That is why I
prefer to leave these matters to the organisational theorists (despite
their insistence upon using a language whose syntax and vocabulary is
identical with those of English but whose semantics appears to come from
Alpha Centauri).

Ten years ago, some colleagues in Braunschweig compared analyses of the
same accident (the Brühl derailment) using WBA and using STAMP. STAMP
identified a lot of organisational features of the Deutsch Bahn (German
railways, as it then was; now it's DB). STAMP likes to see feedback, but
the DB, like many German organisations, is hierarchical and STAMP wanted to
see cycles where things were acyclic. It wouldn't have been helpful,
because, well, I guess you could try to tell DB to change things, but they
would say "we have been doing it like this for over a century; here are the
reasons we do it this way (giving you the very thick history book); it has
evolved so and it more or less works; and if we change it to something new
we are likely to introduce weaknesses which we won't know about until we
start having accidents because of them." And, you know, that's not a bad
set of reasons: you don't change things that aren't really broken, even
when a major scientist redefines "broken" for you. (In contrast, they are
happily adopting WBA through third-party recommendation and training.)*


On Wed, Oct 9, 2013 at 7:52 AM, Peter Bernard Ladkin <
ladkin at rvs.uni-bielefeld.de> wrote:

> On 10/8/13 7:43 AM, Matthew Squair wrote:
>
>> Isn't the question of whether you trust their efforts really a variant of
>> the agency dilemma? And
>> isn't that what 'design' of the socio-technical system should address,
>> and what a methodology such
>> as STAMP can assist you in doing?
>>
>
> Well, I bet some System Safety List old hands are chuckling to themselves
> at this one. I'd hate to disappoint........
>
> Let's stick with accident analysis. We have our own method for causally
> analysing accidents, called WBA. It is used in certain large German
> companies, and of course Causalis uses it for clients who wish for causal
> analyses.
>
> WBA does not have an inbuilt method for classifying
> operator/organisational/**society/legal/governmental (OOSLG) factors as
> such. We use the PARDIA classification for pointy-end activities and use
> decision-theoretic analyses (Rational Cognitive Modelling) for multi-agent
> interactions. WBA does determine where any OOSLG factors are present and
> those that are not addressed by PARDIA and RCM I like to leave for the
> organisational scientists such as Downer and Perrow.
>
> Two hundred fifty years ago, David Hume proposed two characterisations of
> cause which have persisted. One is the constant-conjunction criterion
> (CCC), beloved of those who collect statistics on repeatable events, for
> example the superb recent work of Judea Pearl and colleagues. The other is
> the counterfactual criterion, which WBA uses in a form called the
> Counterfactual Test (CT). Almost all conceptions of causality in the
> scientific, engineering and philosophical literature are one or the other
> of these. CT is used in an intuitive fashion by more or less all aviation
> accident investigations, and it occurs explicitly in the USAF guidance for
> aviation accident investigation. It should be more or less obvious why this
> is so - commercial-aviation accidents are not events which repeat in a
> manner in which CCC can address. (In contrast, CCC *obviously* helps with
> road safety, because road accidents happen with appropriate frequencies for
> CCC techniques to be useful.)
>
> Ten years ago, Nancy announced that she had a new conception of causality,
> which was embodied in STAMP. I saw, and continue to see, a problem in
> redefining a concept which has had a good and productive run in science for
> three centuries (if not two and a half millenia). It could well be that
> this new concept is a very useful concept; it could be very helpful in
> identifying areas of interest in accident investigation; indeed, judging by
> the interest in STAMP I imagine many people think it is. But why not choose
> a different word for it?
>
> We had a discussion on the York list. It wasn't scientifically very
> fruitful (but I do remember fondly - and repeat - a particular piece of
> repartee).
>
> People who like STAMP could *obviously* use WBA for parts of what they do
> - the two methods are compatible. And they would see the same advantage as
> other WBA users. The only hindrance to such practice is use of the word
> "causal" for two different concepts.
>
> The other thing about using STAMP is you have to buy the model. Now, I'm
> sure it is helpful, because the people developing STAMP are very smart and
> very dedicated and have been at it for a decade. But is the model right?
> One might well be able to persuade engineers that the STAMP
> social/organisational model is the bee's knees, but it is a quite different
> matter to persuade the experts in those things, the organisational
> scientists.
>
> Constance Perrin wrote a book in which she investigated some incidents at
> nuclear power stations and came to the conclusion that there was a tension
> between the way the plant was conceived to work organisationally and the
> architecture of plant operations impregnated in the minds of the operators,
> who came mostly from the "nuclear navy", which had/has a modus operandi
> completely different from the intended plant-operations architecture. A
> crucial insight. It is not obvious to me how a STAMP analysis would lead
> you to the same conclusion. (Maybe a good project to try?) That is why I
> prefer to leave these matters to the organisational theorists (despite
> their insistence upon using a language whose syntax and vocabulary is
> identical with those of English but whose semantics appears to come from
> Alpha Centauri).
>
> Ten years ago, some colleagues in Braunschweig compared analyses of the
> same accident (the Brühl derailment) using WBA and using STAMP. STAMP
> identified a lot of organisational features of the Deutsch Bahn (German
> railways, as it then was; now it's DB). STAMP likes to see feedback, but
> the DB, like many German organisations, is hierarchical and STAMP wanted to
> see cycles where things were acyclic. It wouldn't have been helpful,
> because, well, I guess you could try to tell DB to change things, but they
> would say "we have been doing it like this for over a century; here are the
> reasons we do it this way (giving you the very thick history book); it has
> evolved so and it more or less works; and if we change it to something new
> we are likely to introduce weaknesses which we won't know about until we
> start having accidents because of them." And, you know, that's not a bad
> set of reasons: you don't change things that aren't really broken, even
> when a major scientist redefines "broken" for you. (In contrast, they are
> happily adopting WBA through third-party recommendation and training.)
>
> All of which is not to say that we indulge heavily in NIH round here.
> Indeed, there was a major STAMP workshop recently put on by colleague
> Schnieder in Braunschweig, which generated a lot of interest. That's very
> welcome - as Nancy says, the important thing is thinking hard about hazards
> and accidents and using whatever help you can get.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Faculty of Technology, University of
> Bielefeld, 33594 Bielefeld, Germany
> Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de
>
>
>
>
>
> ______________________________**_________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-**Bielefeld.DE<systemsafety at TechFak.Uni-Bielefeld.DE>
>



-- 
Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142

Telephone: 617-258-0505
Email: leveson at mit.edu
URL: http://sunnyday.mit.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20131010/46faa944/attachment-0001.html>