[SystemSafety] Analysis of some Work Breakdown Structure projects
Derek M Jones
derek at knosof.co.uk
Wed Jun 9 13:16:15 CEST 2021
Martyn,
The 2106.03679 paper data derives from the Watts Humphrey's work
at CMU. There is lots more to be found in this data, and the
paper is a start.
>> Defect per KLOC is meaningless unless it is connected with usage
>> data, e.g., there can be zero defects per KLOC (because the software
>> has no users), or lots per KLOC because it has millions of users.
>
> The datasets from http://arxiv.org/abs/2106.03679 that you analysed contain defects injected and defects found later in
> development and repaired. have you analysed those?
This is the data behind figure 6.42.
But there is no usage data.
>> I've never seen a breakdown by individual. It's possible to do, when
>> mining github (actually this is by user id, and there are cases of
>> the same person having multiple ids), but again usage needs to be
>> taken into account.
>
> Again, the http://arxiv.org/abs/2106.03679 data seems to show individuals. The Watts Humphrey study below does that too.
Yes, tasks are associated with individuals.
But again, no usage data.
>>> There was data of this sort from the SEI 30 years ago and some from UK MoD, and some reports by the CHAOS group
>>> twenty years ago but nothing I know of recently.
>>
> The SEI data I referred to was from a study carried out by Watts Humphrey, of the Software Engineering Institute at
> Carnegie-Mellon University, analysed the fault density of more than 8000 programs written by 810 industrial software
> developers. resources.sei.cmu.edu/asset_files/SpecialReport/2009_003_001_15035.pdf p132
Thanks for the link. I had not seen this collection of Watts Humphrey
columns before.
The column name of table 25-1 should read:
"Average detected defects per KLOC".
The question is then: How much effort was put into
detecting defects?
The metric Defects_per_KLOC only makes sense when effort
to detect the defacts is taken into account.
I can create programs with zero Defects_per_KLOC, simply by
putting zero effort into detecting defects.
>> UK MoD? This does not ring any bells for me. Do you have a reference,
>>
> My reference was to the analysis of Boeing flight control software published in Crosstalk
> German, A.: Software static code analysis lessons learned. Crosstalk
> 16(11) (2003)
Thanks for the reference.
Table 1 lists Anomalies per lines of code.
But again, no indication of the effort involved in detecting
those anomalies.
> and to the review of the Full Authority Digital Engine Controller that was installed in Chinook helicopters; which is
> described in a House of Commons report into the Mull of Kintyre Chinook accident on 2 June 1994 . This said:/In the
I will have a look at this, but I suspect that effort to detect data is
not included.
> summer of 1993 an independent defence IT contractor, EDS-SCICON, was instructed to review the FADEC software; after
> examining only 18 per cent of the code they found 486 anomalies and stopped the review/.
Did they record the effort (I imagine their time), needed
to detect each anomaly? This kind of data is rare.
--
Derek M. Jones Evidence-based software engineering
tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com
More information about the systemsafety
mailing list