[SystemSafety] Agile methods

[Derek M Jones]

> Given the sad state of medical device software, it does not seem that
> simply using Agile methods proves anything. Homa Alenzadeh, a grad student
> at UIUC, has looked at medical device recalls. The following is from an
> abstract she wrote for a talk she gave: "Medical device incidents are one

Paper + spreadsheet listing 1,211 recall events:
http://users.crhc.illinois.edu/alemzad1/pub.html

> of the leading causes of serious injury and death in the United States.
> Between years 2006 and 2011, 5,294 recalls and approximately 1.2 million
> adverse events of medical devices were reported to the US Food and Drug
> Administration (FDA). Almost 23 percent of these recalls were due to
> computer-related failures, of which approximately 94 percent presented
> medium to high risk of severe health consequences (such as serious injury
> or death) to patients.   '
>
> If my arithmetic is correct (1,200,000*.23 and then divide by 5 then
> multiply by .94), it appears from Homa's data that there are 51,888
> reported potentially serious incidents to the FDA regarding software each
> year. That is 144 every day. And many incidents are not reported (the
> reporting rules are pretty sketchy) and that is only in the U.S.
>
> The simple fact that mdedical device manufacturers have used Agile methods
> does not prove anything about their efficacy for safety-critical systems
> given the sad state of safety in that industry.
>
> Nncy
>
>
> On Mon, Sep 2, 2013 at 12:40 PM, Steve Tockey <Steve.Tockey at construx.com>wrote:
>
>>   All,
>> I have personal experience using Agile, and personal experience doing
>> safety-critical development (DO-178B), but not personal experience doing
>> agile development for a safety critical system. Instead of being polarizing
>> and claiming "it can't be done", I'll simply say that I believe that it is
>> possible to do it—it could just end up being very costly to do so. The
>> worst case would be developing the code for the system in an truly agile
>> way, and then looping back and re-developing it with the necessary safety
>> analysis, etc. in place using more traditional processes.
>>
>>   All that said, I have a report titled, "AAMI
>> TIR45: 2012 Technical Information Report      Guidance on the use of AGILE
>> practices in the development of medical device software". AAMI is the
>> "Association for the Advancment of Medical Instrumentation" and they're
>> based at
>> 4301 N. Fairfax Drive, Suite 301
>> Arlington, VA   USA    22203-1633
>> www.aami.org
>>
>>   The report abstract states, "Over the past several years, AGILE software
>> development has become an accepted method for developing software products.
>> There have been questions from both manufacturers and regulators as to
>> whether (or which) AGILE practices are appropriate for developing medical
>> device software. Enough medical device manufacturers have implemented AGILE
>> practices in their software development so that answers to these questions
>> can be documented. Having clear guidance of which practices have been found
>> to be appropriate will be very useful for all developers of medical device
>> software. This TIR will provide recommendations for complying with
>> international standards and U.S. Food and Drug Administration (FDA)
>> guidance documents when using AGILE practices to develop medical device
>> software."
>>
>>   A preview copy is at:
>>
>> http://marketplace.aami.org/eseries/scriptcontent/docs/Preview%20Files/TIR45_1208_preview.pdf
>>
>>   I have a licensed copy, but it's copyrighted material and can't be
>> shared. If you're really interested in the content, you'll have to get a
>> copy on your own (sorry).
>>
>>   Having said that, three comments about all this:
>>
>>   1) I don't know what sort of status AAMI has with respect to FDA policy
>> on medical device software development. In other words, I don't know if the
>> FDA has actually agreed to this, or it's simply AAMI's suggestion on how
>> one might approach it.
>>
>>   2) The AAMI approach talks about agile development in a way that's very
>> consistent with how most competent people would characterize agile
>> development.
>>
>>   3) I took a look at the Bruce Douglass report that Geoffrey Biggs
>> mentioned (Thanks, Geoffrey)
>>
>> http://www.ibm.com/developerworks/rational/library/agile-analysis-practices-safety-critical-development/
>>   I've worked with Bruce before, and I have a lot of respect for his work.
>> However, I'd be willing to bet that most people who really understand what
>> agile development is could easily argue that the Harmony method discussed
>> by Bruce would not be considered agile. It's clearly iterative, but
>> iteration alone does not constitute true agility. *I* wouldn't consider
>> the Harmony method described by Bruce to be truly agile.
>>
>>
>>   -- steve
>>
>>
>>
>>    From: René Senden <rene.senden at gmail.com>
>> Date: Monday, September 2, 2013 8:32 AM
>> To: 'M Mencke' <menckem at gmail.com>
>> Cc: "systemsafety at techfak.uni-bielefeld.de" <
>> systemsafety at techfak.uni-bielefeld.de>
>>
>> Subject: Re: [SystemSafety] Agile methods
>>
>>    Myriam,****
>>
>> ** **
>>
>> You are right about my initial question being somewhat polarizing, perhaps
>> because I tacitly assumed that anyone who’d answer this question with
>> “Yes”, would also include some****
>>
>> of the corresponding experiences … Regarding experience with
>> reconciliation of agile environments with typical/common/… (software)
>> safety standards, I am very interested in the ****
>>
>> development processes and corresponding results/evidence/outputs, all this
>> should be auditable. Is it (at all) possible to harmonize these very
>> different worlds, or would any such ****
>>
>> attempt result in compromising either? Transparency, auditability,
>> documentation, and safety specific processes such as analyses are examples
>> that, to me, seem to be particularly ****
>>
>> difficult to address in an agile environment, let’s take “scrum” as an
>> example of a popular agile approach.****
>>
>> ** **
>>
>> Rene****
>>
>> ** **
>>
>> ** **
>>
>> *From:* systemsafety-bounces at lists.techfak.uni-bielefeld.de [
>> mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de<systemsafety-bounces at lists.techfak.uni-bielefeld.de>]
>> *On Behalf Of *M Mencke
>> *Sent:* maandag 2 september 2013 14:39
>> *To:* Jon Davies
>> *Cc:* systemsafety at techfak.uni-bielefeld.de
>> *Subject:* Re: [SystemSafety] Agile methods****
>>
>> ** **
>>
>> René,****
>>
>> I have come across a situation in the railway field where an
>> organization was required by the customer to comply with a safety standard
>> which has specific requirements on the software development process, in
>> this case, EN 50128, where this was previously not required.  ****
>>
>> It seems to me that this type of situation typically arises in
>> industries/products where the SIL concept has traditionally not been
>> applied, or is being introduced. The customer creates a “SIL requirement”
>> as a type of marketing argument, where the SIL is understood by the
>> customer as applying to equipment rather than to a function (in order to be
>> able to state “This equipment is SIL X” in commercial scenarios).****
>>
>> Regarding your question, “Have you encountered a situation, in industrial
>> practice, in which an organization developing software following an agile
>> methodology has to comply with a safety standard which has specific
>> requirements on the software development process?” ****
>>
>> This is a polar question, which can only be answered with a yes/no. ****
>>
>> If you are interested in practical experience, it might be useful to add
>> to this question which aspect(s) of such a situation you are interested in.
>> For example, if the answer is “yes”, was the project actually a success?
>> Which measures/procedures were implemented in the company to approach this
>> problem and reconcile the development process with such requirements?, etc.
>> ****
>>
>> Depending on the type of information you are seeking, it may not be
>> possible to provide particular details regarding such a state of affairs,
>> as this information may be company-specific.****
>>
>> Regards,****
>>
>> Myriam.****
>>
>> ** **
>>
>> 2013/9/2 Jon Davies <jdavies at theiet.org>****
>>
>> On 30 August 2013 18:02, René Senden <rene.senden at gmail.com> wrote:
>>> Dear all,
>>> ****
>>
>>> Do any of you have practical experience with reconciling established
>> agile
>>> software development with software safety requirements (e.g. IEC-61508 or
>>> DO-178..) ?****
>>
>> Yes, and we usually end up throwing away the software developed using
>> "agile" methods, and starting again properly.
>>
>> I'm taking "agile software development" as meaning the development of
>> software using processes consistent with the agile manifesto:
>> http://agilemanifesto.org/ - to quote the relevant part:
>> "...we value... working software over comprehensive documentation"
>>
>> this is fundamentally in conflict with many of the things we know
>> about building high integrity software, and so "agile" methods are
>> fundamentally in conflict with developing software for safety critical
>> systems.
>>
>> There's plenty to learn from agile development methods that might be
>> useful in high integrity software development, but that's a whole
>> different discussion.  Every time we discuss agile development here,
>> we end up back at the need to use a development process that builds in
>> correctness - we can't test exhaustively, so we need a process that
>> builds integrity in.  Agile methods don't do this.
>>
>> Jon****
>>
>> _______________________________________________
>> The System Safety Mailing List
>> systemsafety at TechFak.Uni-Bielefeld.DE****
>>
>> ** **
>>
>> _______________________________________________
>> The System Safety Mailing List
>> systemsafety at TechFak.Uni-Bielefeld.DE
>>
>>
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>

-- 
Derek M. Jones                  tel: +44 (0) 1252 520 667
Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
Software analysis               http://www.knosof.co.uk