[SystemSafety] Analysis of some Work Breakdown Structure projects

[Derek M Jones]

Martyn,

All figure refer to my Evidence-based book: http://knosof.co.uk/ESEUR/

> I'd be interested in data on the defects injected and fixed.  How many per KLOC, how variable between individuals, what

Defect per KLOC is meaningless unless it is connected with usage
data, e.g., there can be zero defects per KLOC (because the software
has no users), or lots per KLOC because it has millions of users.

Usage data is extremely rare.
Figure 6.4 shows fault reports against package installations.

I've never seen a breakdown by individual.  It's possible to do, when
mining github (actually this is by user id, and there are cases of
the same person having multiple ids), but again usage needs to be
taken into account.

> relationship to component size (LOC), what percentage injected in each phase, where they were found, how long a defect

I know of two datasets giving the phase in which a mistake was made
and the phase it was detected in; the tiny dataset figure 6.41.
The time to fix does increase with phase separation, as seen in
figure 6.42 (this is data from CMU).

Cost to fix must increase with phase separation, because each
phase involves work.  But what about the cost of detection.; this
rarely gets mentioned, and it may be cheaper to detect some mistakes
in later phases.  I have no data on this, but see: Figure 6.55
Management want to minimise cost_to_detect plus cost_to_fix.

 > typically remains before being found and fixed

or never fixed because the code is rewritten/deleted:
Figure 11.80

> typically remains before being found and fixed ... Anything of this sort would be interesting. There was data of this 
> sort from the SEI 30 years ago and some from UK MoD, and some reports by the CHAOS group twenty years ago but nothing I 
> know of recently.

"The Leprechauns of Software Engineering" does a good job of
showing how most of this 30+ year old folklore is not evidence based.
https://leanpub.com/leprechauns

UK MoD?  This does not ring any bells for me.  Do you have a reference,
or are they just parroting US folklore?

> Martyn
> 
> On 08/06/2021 11:07, Derek M Jones wrote:
>> All,
>>
>> Until recently data on the use of Agile in safety critical software
>> was almost non-existent.
>>
>> I'm not sure if the use of work breakdown structure counts as
>> Agile, but here is an analysis of lots of data (project 615
>> was one of the safety critical projects):
>> http://arxiv.org/abs/2106.03679
>>
>> Never having used WBS, I'm not sure of the kind of questions
>> that those involved might ask about such data.  Suggestions
>> welcome.
>>
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety

-- 
Derek M. Jones           Evidence-based software engineering
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com