[SystemSafety] Fwd: Contextualizing & Confirmation Bias

Wed Feb 5 20:56:04 CET 2014

PBL writes:
>The term "confirmation bias" originally refers to the following
phenomenon. (1) You believe A. (2)
>You act as if A. (3) You experience the world as if A is true. (4) You end
up believing A more than
>you did at the beginning simply because of that.

>Now, that's a fairly well-circumscribed phenomenon, and it is a lot more
limited than what has been
>proposed here.

Peter, this is not the definition of confirmation bias used in the
psychology literature. It appears that you have made up a very
circumscribed definition, and then conclude that the phenomenon is limited
and circumscribed because your definition was.

It would be helpful if you would provide references if you are going to
provide definitions that differ greatly from the widely accepted.

Nancy

On Wed, Feb 5, 2014 at 2:48 PM, Peter Bernard Ladkin <
ladkin at rvs.uni-bielefeld.de> wrote:

> I worried when I replied to Derek that I was opening a can of worms. So
> let me first avoid answering
> Derek's and Nick's questions and lay some groundwork. Under the caveat
> that I am not sure how much
> of this applies to system safety, but under the conviction (bias?) that a
> lot less of it applies
> than people have suggested.
>
> So, let me apologise in advance to those who think this is peripheral.
> Since it has been cited as a
> reason why "safety cases" in the broad sense are to be trusted less, I do
> think that some discussion
> is not untoward.
>
> The term "confirmation bias" originally refers to the following
> phenomenon. (1) You believe A. (2)
> You act as if A. (3) You experience the world as if A is true. (4) You end
> up believing A more than
> you did at the beginning simply because of that.
>
> Now, that's a fairly well-circumscribed phenomenon, and it is a lot more
> limited than what has been
> proposed here. It doesn't apply to safety cases in the way that has been
> suggested, indeed part of
> it seems to be an entirely desirable phenomenon: if we experience the
> world as if (3) the safety
> case is true, then we have achieved the goal the system development was
> intended to achieve! (4) is
> utterly unimportant! So even the question of confirmation bias just
> doesn't seem to arise.
>
> The vicar lives just up the hill from me. The church is just downhill from
> me. Just before the
> church, there is a main street, in the form of a big curve, which is
> sparsely but regularly used,
> with a 30kph limit with which few drivers conform. Sunday service starts
> at 10am. Often, he comes
> barrelling down the road on his kid's mountain bike at 2 minutes to 10 -
> the sermon has taken just a
> little longer to finish as he'd hoped. I know my bicycles - his brakes
> aren't that good. This is the
> only time - I swear - that I pray, briefly. For I am not a Christian. But
> I like him a lot and he
> does wonderful work.
>
> He's still alive; he still does it. I mention it to him every so often.
>
> The point is that there has never been a car exactly there, going too
> fast, on a Sunday morning at
> exactly the wrong time. But I think there could be, so I call his behavior
> confirmation bias.
>
> Now, for all I know, he might think he has "higher protection". If so, he
> might call the results
> "confirmation". I call them "confirmation bias" because I don't think he
> does. Whether you judge it
> to be a bias or not depends on the assumptions you make about the world.
>
> Millikan was acting as if it were true that there is just one number (in
> suitable units) which is
> the charge on the electron, and he was out to find it. An appropriate
> charge of "confirmation bias"
> would attach to the phenomenon that his results suggest there is indeed
> just one number/unit which
> is The Charge Of The Electron. But I don't call it a bias, and neither
> should Derek or Nick, because
> we all believe it (I take it). And that is not what either Derek or Nick
> were suggesting was improper.
>
> Similarly, consider my continually recalculating "2+2". I mostly get the
> answer 4, and when I don't,
> I look for what got in the way. That is not confirmation bias. 2+2 is
> indeed 4; there are no two
> ways about it; it's not an assumption I make about the way the world might
> be. I can prove it from
> Peano arithmetic, and if you say (as some psychologists do), "oh, that's
> just because you're used to
> thinking that way", I would reply "anyone who doesn't get the answer 4
> either doesn't understand
> what 2 is, or doesn't understand the operation +". Here, of course, we
> open the whole
> Kripke-on-Wittgenstein-on-Rules can of worms. So let me not.
>
> In the specific explanation of the meaning of the term above,
> "confirmation bias" only applies to an
> argument if someone acts as if the argument were true. Which, when the
> argument is a safety case and
> the safety case has been accepted, people do. All people. But I propose it
> is only helpful to call
> it a bias if it doesn't accurately represent the world as it is (yes, lots
> of metaphysical
> assumptions here). Indeed, there is a considerable worry that most safety
> cases do not present a
> very accurate representation of the world as it is, especially when we are
> proposing that events
> round about the systems have objective frequencies. And when we accept one
> for reasons peripheral to
> its own partial validity, then it is appropriate to call it a bias. And
> when we accept it because,
> well, that's what's down on paper, and there's a lot of it, and boy that
> was a huge amount of work
> by dedicated and honest people, rather than on its objective merit of
> establishing what it claims,
> then that is a bias worthy of a name. I wouldn't pick the name
> "confirmation bias" because that's
> already taken, but I can understand the temptation.
>
> What Nancy was talking about is the following (possible) phenomenon.
> Suppose someone presents you
> with an argument to a conclusion. The argument is not a rigorous
> deduction; and it makes lots of
> assumptions, implicit and explicit. Do you believe it, or are you
> sceptical? Well, when that
> argument is hundreds of pages of all kinds of varied considerations long,
> and you don't understand
> half of it because you are not as expert in the math as those who wrote
> it, and the conclusion is
> that "this system is acceptably safe in operation", then belief seems to
> be close to a matter of
> faith. This is especially so, since argument critics are almost inevitably
> right. But some systems
> nevertheless do turn out to be acceptably safe in operation, so the
> sceptical inference from "this
> argument for acceptably-safe operation is full of holes" to "this system
> is not acceptably safe in
> operation" is not invariably correct; indeed is often wrong.
>
> The phenomenon to which Nancy was drawing attention is that, in a
> safety-case regime, the conclusion
> "this system is acceptably safe in operation" is somehow more likely to be
> erroneously drawn than in
> a regime where specific prophylactic measures are applied during
> development and ticked off as
> fulfilled. In the US. In the oil industry.
>
> Well, OK, maybe that's so. But I am not really OK with using the term
> "confirmation bias" to suggest
> a reason for that. I'm a "confirmation bias fundamentalist". I think you
> should use the term in its
> original use.
>
> On 2/5/14 6:40 PM, Derek M Jones wrote:>
> > So its not confirmation bias if, sometime after doing the
> > analysis, you are shown to be correct?
>
> I didn't say that. But I would say that an attribution of confirmation
> bias is much harder to
> sustain if you succeeded in reaching the truth. Indeed, I'd say that an
> attribution of any bias at
> all is hard to sustain if you succeed in reaching the truth, except if
> it's due to happenstance.
>
> > I have always regarded confirmation bias as not depending on the answer
> > given and actual answer being correlated in any way.
>
> Sure it is.
>
> A die is an instrument with a purpose: when you roll it "randomly", then
> any face is as likely as
> any other to be shown. When one face is more likely that some other, we
> call the die "biased". The
> vicar is not suffering from confirmation bias if indeed he is "protected
> by higher agencies". He is
> just acting according to the way the world is. Whereas, if there is no
> higher agency protecting him
> as he approaches the junction, then his actions can be said to be
> partially due to "confirmation bias".
>
> The STS people want a term for a phenomenon independent of the way the
> world "actually is". I am
> sceptical whether one can even reasonably individuate the phenomenon when
> one makes no assumptions
> about the way the world really is.
>
> On 2/5/14 6:45 PM, Nick Lusty wrote:
> > Whilst accepting that all science is done in its own context, surely
> getting "the right answer ...
> > when there was a right answer to get" is a not evidence of the absence
> of confirmation bias in the
> > experiment.
>
> Neither is it evidence of the absence of elephants in the room. I have no
> idea whether Millikan's
> results suffered in some way from confirmation bias. I am only suggesting
> that the evidence we have
> does not indicate that they did.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Faculty of Technology, University of
> Bielefeld, 33594 Bielefeld, Germany
> Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>

-- 
Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142

Telephone: 617-258-0505
Email: leveson at mit.edu
URL: http://sunnyday.mit.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20140205/71e22ade/attachment-0001.html>