[SystemSafety] Fwd: Measurement + Control
Matthew Squair
mattsquair at gmail.com
Sun Dec 15 22:29:51 CET 2013
Over the years I've become more and more cautious, and sceptical, about
safety cases as they've grown outside their original context and into more
general usage.
So, compare and contrast, having had the privilege of reviewing a few
safety assessment reports developed under MIL-STD-882 by US contractors
they've invariably stated something like, 'all hazards identified in this
report have been eliminated or controlled...' rather than the system was
'safe' or that safety "risk had been reduced to 'so far as is reasonably
practical'". I think there's some hard nosed realism, and wisdom, in that
approach.
Safety cases IMO are inherently vulnerable to the narrative fallacy effect
where a good news story is fitted to all that data telling you things are
fine, but actually it's the rare outlier extreme events that dominate the
safety and for which we have little to no data.
Then there's the framing effects that such documents/arguments engender
both in terms of looking at it as an argument that the system is safe with
all the dangers of confirmatory bias and also that if the issue is not in
the case, well the issue falls off the edge of the table.
As currently practiced I think that despite all the good intentions they're
an inherently flawed concept, and we should just stop. That goes for the
regulators too.
I wrote a little bit more about the problem of safety cases here
http://wp.me/px0Kp-1Fu.
On Mon, Dec 16, 2013 at 6:15 AM, Andrew Rae <andrew.rae at york.ac.uk> wrote:
> Absolutely. I think both supporters and opponents of safety cases can
> agree that demonstrating that a system is safe is not the goal of safety
> engineering.
> Even going into a program with the _mindset_ that your job is to prove
> that the system is safe is asking for trouble.
>
> I do think that having an external body/person with a somewhat adversarial
> mindset is important for good safety practices. It's the only real way to
> distinguish between _thinking_ that you're managing safety well, and
> actually managing it well. This necessarily means that some of the safety
> activity is going to involve _demonstrating_ safety, rather than improving
> it, but as Nancy says, this should be much more straightforward if you've
> actually made a safe system. [Slight caveat - being safe and being
> demonstrably safe are not always the same thing. It is worth thinking early
> in the program about how you are going to demonstrate safety].
>
> There are good reasons to believe that making your reasons for believing
> that a system is safe explicit will help to combat the inevitable
> confirmation bias that goes along with any demonstration of safety. That
> applies whether your reasons are related to a particular set of practices,
> or following a standard, or something particular to your program and system
> architecture. I understand why Nancy disagrees with this, and I wish there
> weren't so many cases of "retrospective" application of goal-based
> standards to support her position. "But that's not proper use of safety
> cases" becomes a weaker cry every time someone does it and calls it a
> safety case.
>
> As an aside, and picking two particular notations at random, I'd really
> like to see someone not already involved with STPA or GSN produce and have
> certified/accepted a system using both at once, just to make clear that
> designing a safe system and demonstrating its safety are complementary
> rather than competing ideas.
>
> My system safety podcast: http://disastercast.co.uk
> My phone number: +44 (0) 7783 446 814
> University of York disclaimer:
> http://www.york.ac.uk/docs/disclaimer/email.htm
>
>
> On 15 December 2013 18:35, Nancy Leveson <leveson.nancy8 at gmail.com> wrote:
>
>> I am getting increasingly frustrated by a prevalent attitude that the
>> goal of safety engineering is to prove that a design is safe. I am not
>> picking on Drew -- he is bringing up a good point. But it emphasizes the
>> absurdity of the approach if safety is being "outsourced."
>>
>> The goal of safety engineering is to design safe systems. It is not to,
>> after-the-fact or independently, try to show that a system is safe. At
>> best, the latter goals are simply add-ons to the primary goal, i.e., a
>> final step that is used simply to ensure that what was done before is
>> approved). If safety engineering is done correctly, i.e., the hazard
>> analysis and safety engineering steps have been accomplished by the
>> engineers as they are designing the system and making design decisions,
>> then the after-the-fact preparation of the case for the regulators is
>> simple and consists of simply packaging up what was done during
>> development.
>>
>> Engineering is not about making "arguments."
>>
>> Nancy
>>
>>
>>
>> On Thu, Dec 12, 2013 at 3:48 AM, Andrew Rae <andrew.rae at york.ac.uk>wrote:
>>
>>> Thanks for alerting to the availability of the article, Peter.
>>>
>>> One of the "outsourcing" issues which the paper alludes to but does not
>>> elaborate on is outsourcing of the safety function itself.
>>> I've found it hard to get exact figures, but anecdotal and advertising
>>> indicators suggest that outsourcing major hazardous facilities safety
>>> reports
>>> (COMAH in the UK) to consultants seems to be a widespread practice.
>>> Combine this with overworked and under-staffed regulators, and we have
>>> safety analysis prepared by consultants and then never thoroughly
>>> reviewed by either the customer or the regulator. The Buncefield safety
>>> report
>>> had been in the system for well over a year without the review being
>>> completed at the time of the accident.
>>>
>>> I worry sometimes that safety expertise can be too intimidating,
>>> damaging culture. Because the safety experts see themselves as the only
>>> ones competent to do the analysis, everyone else sees safety as something
>>> to be outsourced to internal or external expert groups. The result is that
>>> we have a small group of people doing safety work (themselves too divorced
>>> from the actual design and operation of the system to do the work
>>> properly), and no one spare to review and check the work, or to improve the
>>> way the work is done.
>>>
>>>
>>> My system safety podcast: http://disastercast.co.uk
>>> My phone number: +44 (0) 7783 446 814
>>> University of York disclaimer:
>>> http://www.york.ac.uk/docs/disclaimer/email.htm
>>>
>>>
>>> On 11 December 2013 18:27, Peter Bernard Ladkin <
>>> ladkin at rvs.uni-bielefeld.de> wrote:
>>>
>>>> I received this message from the publishing house Sage. They used to be
>>>> known mostly for social-scince stuff, but recently took over PEP, who
>>>> do/did the various parts of the Proceedings of the IMechE, and I suppose
>>>> those for other engineering professional societies also.
>>>>
>>>> Sage offers free access to certain articles until Sunday, somif you're
>>>> quick you can download the Buncefield article by Colin Howard, as I did.
>>>> The Secretary of the German functional safety standards committee, Ingo
>>>> Rolle, is interested in issue concerning Buncefield, and I sent it to him.
>>>> He is on this list, as I seem to remember is Carl Sandom, who is convening
>>>> the IEC project on human factors in functional safety. Ingo's comment
>>>> speaks directly to HF issues.
>>>>
>>>> [begin comment Ingo Rolle]
>>>>
>>>> Thanks for this article, one of the few I read in full recently. The
>>>> switching device for the high level alarm was replaced in 2004, prior to
>>>> the accident, at that tank which was overfilled in 2005. I draw the
>>>> conclusion that the change of switching principle in that device was a
>>>> major contribution to the ill-fated sequence of events, although I didn't
>>>> find a more explicit statement of the author on this.
>>>>
>>>>
>>>> [I agree with that conclusion. PBL]
>>>>
>>>>
>>>>
>>>> Nevertheless, the scenario described of actions and efforts to replace
>>>> the switch and amend the situation reminded me of my own experience as an
>>>> engineer associated with process facilities. In my view such a situation
>>>> [as occurred here] is the clear consequence of outsourcing of maintenance
>>>> personnel. It is strange to make large investments, establish facilities on
>>>> green field sites, and then leave such assets without people assigned to
>>>> operate them and look after them properly. I think we are the first
>>>> civilization to engage in such foolishness ("outsourcing"). Only people who
>>>> know and understand the facilities, have a close relation to them, know
>>>> their colleagues and environment and have some time for that will track
>>>> weaknesses, address them and keep the whole thing safe and well running.
>>>>
>>>>
>>>>
>>>> One may outsource particular activities but the core management of
>>>> maintenance must stay in the company or facility
>>>>
>>>>
>>>>
>>>> Ingo Rolle
>>>>
>>>>
>>>>
>>>> [end comment Ingo Rolle]
>>>>
>>>>
>>>> PBL
>>>>
>>>> Prof. Peter Bernard Ladkin, University of Bielefeld and Causalis Limited
>>>>
>>>> Begin forwarded message:
>>>>
>>>> *From:* "SAGE Measurement and Control" <
>>>> announcements at news.sagepub.co.uk>
>>>> *Date:* 4 December 2013 14:21:13 CET
>>>> *To:* <ladkin at rvs.uni-bielefeld.de>
>>>> *Subject:* *Measurement + Control*
>>>> *Reply-To:* "mailbox20825x12699" <mailbox20825x12699 at news.sagepub.co.uk
>>>> >
>>>>
>>>> Submit your research to Measurement + Control View this online<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=1&e=2&x=2456997.0>
>>>> | Forward to a friend<?subject=Exclusive+Content+from+SAGE&body=Your+friend+thought+you+might+be+interested+in+this+message+from+SAGE.+Click+here+to+view+the+content:+http://www.sagepub.co.uk/email/online/2013/3C11.htm+If+this+content+is+of+interest+to+you+please+visit:+http://www.sagepub.com/emailAlerts.sp+to+sign+up+to+our+e-alerts+and+we+will+update+you+with+new+books+and+journals+relevant+to+your+subject+area.>
>>>> | Email alerts<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=3&e=2&x=2456997.0> [image:
>>>> Measurement and Control]<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=4&e=2&x=2456997.0> Is
>>>> Measurement + Control the right journal for your research?
>>>> ------------------------------
>>>> *Measurement + Control* (MAC) is a peer-reviewed journal now
>>>> published ten times a year by SAGE. The journal publishes practical and
>>>> technical research and news pieces from both the science and engineering
>>>> industry and academia.
>>>>
>>>> *[image: »]* *Find out more
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=4&e=2&x=2456997.0>*
>>>> *The benefits of publishing in this journal*
>>>> ------------------------------
>>>>
>>>>
>>>> - *Peer review of your research*
>>>> - *Raise your individual profile as well as your company*
>>>> - *Efficient publishing* *of your paper* through SAGETrack, the
>>>> online submission, revision and progression system
>>>> - *Broad readership* – *Measurement + Control* is disseminated to a
>>>> wide community of experts within the field of measurement and control
>>>> including the membership of the Institute of Measurement and Control
>>>> - *High visibility* via our award winning platform, SAGE Journals
>>>> Online- your article will have global reach and high discoverability
>>>> through online search
>>>>
>>>> Click here to go directly to the online submission and peer
>>>> review system powered by ScholarOne Manuscripts™<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=5&e=2&x=2456997.0>
>>>> *Featured articles: Get free access to these most-read articles
>>>> published in Measurement + Control*
>>>> ------------------------------
>>>> SAGE are offering free access to the following articles until
>>>> December 15, 2013
>>>>
>>>> 1. Cable Screens in Hazardous Areas<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=6&e=2&x=2456997.0>
>>>> Chris Towle
>>>>
>>>> 2. Developing Measurement Facilities for Carbon Capture and Storage<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=7&e=2&x=2456997.0>
>>>> Calum Hardie
>>>>
>>>> 3. The Buncefield Incident - 7 Years on: Could It Happen Again?
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=8&e=2&x=2456997.0>
>>>> Colin Howard
>>>>
>>>> 4. Using CFD to Understand Multiphase and Wet Gas Measurement<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=9&e=2&x=2456997.0>
>>>> Neil Barton, Andrew Parry
>>>>
>>>> 5. Pigs, Pipelines and PLUTO: A History of the United Kingdom’s
>>>> Largest Oil Pipeline and Storage System during World War Two
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=10&e=2&x=2456997.0>Tim
>>>> Whittle
>>>>
>>>> I look forward to hearing from you. Please do get in touch with
>>>> any questions about submitting.
>>>>
>>>> Ron Summers
>>>> Editor, *Measurement + Control*
>>>> *r.summers at lboro.ac.uk <r.summers at lboro.ac.uk>* [image: MAC] [image:
>>>> »] View sample issue<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=12&e=2&x=2456997.0>
>>>> [image: MAC]<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=4&e=2&x=2456997.0>
>>>>
>>>> *Editor:*
>>>> Ron Summers
>>>>
>>>> *IMPACT FACTOR:* 0.290
>>>>
>>>>
>>>> HOME<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=13&e=2&x=2456997.0> |
>>>> UNSUBSCRIBE<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=14&e=2&x=2456997.0> | ABOUT
>>>> US<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=15&e=2&x=2456997.0> | PRIVACY
>>>> POLICY<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=16&e=2&x=2456997.0> | CONTACT
>>>> US<http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=17&e=2&x=2456997.0>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> * SAGE Offices: Los Angeles:
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=18&e=2&x=2456997.0>
>>>> 2455 Teller Rd, Thousand Oaks, CA 91320, USA www.sagepub.com
>>>> <http://www.sagepub.com> London:
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=19&e=2&x=2456997.0>
>>>> 1 Oliver's Yard, 55 City Road, London, EC1Y 1SP, UK. Registration No.
>>>> 1017514 www.sagepub.co.uk <http://www.sagepub.co.uk> New Delhi:
>>>> <ena.gordon at sagepub.in> B 1/I 1 Mohan, Cooperative Industrial Area, Mathura
>>>> Road, New Delhi 110 044 India Singapore:
>>>> <http://content.news.sagepub.co.uk/emessageIRS/servlet/IRSL?v=5&a=10050&r=20825&m=12699&l=13&e=2&x=2456997.0>
>>>> 3 Church Street, #10-04 Samsung Hub, Singapore 049483
>>>> www.sagepublications.com <http://www.sagepublications.com>*
>>>>
>>>> MailRef: 3C11 | 18107084 | Julia Young
>>>>
>>>>
>>>> _______________________________________________
>>>> The System Safety Mailing List
>>>> systemsafety at TechFak.Uni-Bielefeld.DE
>>>>
>>>>
>>>
>>> _______________________________________________
>>> The System Safety Mailing List
>>> systemsafety at TechFak.Uni-Bielefeld.DE
>>>
>>>
>>
>>
>> --
>> Prof. Nancy Leveson
>> Aeronautics and Astronautics and Engineering Systems
>> MIT, Room 33-334
>> 77 Massachusetts Ave.
>> Cambridge, MA 02142
>>
>> Telephone: 617-258-0505
>> Email: leveson at mit.edu
>> URL: http://sunnyday.mit.edu
>>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>
--
*Matthew Squair*
MIEAust CPEng
Mob: +61 488770655
Email: MattSquair at gmail.com
Website: www.criticaluncertainties.com <http://criticaluncertainties.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20131216/5453ca24/attachment-0001.html>
More information about the systemsafety
mailing list