<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space; font-family: Calibri, sans-serif; font-size: 14px; color: rgb(0, 0, 0);">
<div><br>
</div>
<div>Derek,</div>
<div><br>
</div>
<div><i>“The rework task information is the same as all the other task information, e.g., date, time, person-id, etc. But there is no field specifying what previous tasks have been reworked.”</i></div>
<div><br>
</div>
<div>You don’t need that for an R% calculation. You only need total time spent on rework divided by total time spent on the project. I looked at CESAW_defect_facts.csv and saw that each row has a project_key column and a defect_fix_time_minutes column. </div>
<div><br>
</div>
<div>CESAW_project_summary.csv has the overall effort data for each project in column “Effort [hours]”.</div>
<div><br>
</div>
<div>Overall R% for the entire data set should be calculate-able by adding up all defect_fix_time_minutes in CESAW_defect_facts.csv and dividing by 60 to convert to hours, then dividing that by the total of the Effort [Hours] column in CESAW_project_summary.csv.</div>
<div><br>
</div>
<div>R% on a project-by-project basis should be calculate-able by adding up all defect_fix_time_minutes in CESAW_defect_facts.csv that have the same project_key and dividing by 60 to convert to hours, then dividing that by the Effort [Hours] value for the row
with the same project_key in CESAW_project_summary.csv.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Martyn,</div>
<div><br>
</div>
<div><i>“<span style="font-family: -webkit-standard;">I'd be interested in data on the defects injected and fixed. How many</span><span style="font-family: -webkit-standard;"> </span><span style="font-family: -webkit-standard;">per KLOC, how variable between
individuals, what relationship to</span><span style="font-family: -webkit-standard;"> </span><span style="font-family: -webkit-standard;">component size (LOC), what percentage injected in each phase, where they</span><span style="font-family: -webkit-standard;"> </span><span style="font-family: -webkit-standard;">were
found, how long a defect typically remains before being found and</span><span style="font-family: -webkit-standard;"> </span><span style="font-family: -webkit-standard;">fixed ...</span>”</i></div>
<div><br>
</div>
<div>Table CESAW_defect_facts.csv has columns including:</div>
<div><br>
</div>
<div>*) wbs_element_key</div>
<div>*) person_key</div>
<div>*) injected_phase_name</div>
<div>*) injected_phase_type</div>
<div>*) defect_found_date</div>
<div>*) defect_removed_phase_key</div>
<div>*) removed_phase_name</div>
<div>*) removed_phase_type</div>
<div><br>
</div>
<div>It seems like you could at least get some of the data you’re looking for, e.g., % injected by phase, % detected by phase, how they were found (failure vs. construction vs. appraisal). You should be able to match the WBS element key in CESAW_defect_facts.csv
to the corresponding task in CESAW_task_fact.csv to get calendar dates for when the original work on the deliverable was done. </div>
<div><br>
</div>
<div><br>
</div>
<div>— steve</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>-----Original Message-----</div>
<div>From: Derek M Jones <<a href="mailto:derek@knosof.co.uk">derek@knosof.co.uk</a>></div>
<div>Organization: Knowledge Software, Ltd</div>
<div>Date: Wednesday, June 9, 2021 at 7:13 PM</div>
<div>To: Steve Tockey <<a href="mailto:Steve.Tockey@construx.com">Steve.Tockey@construx.com</a>>, "<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de">systemsafety@lists.techfak.uni-bielefeld.de</a>" <<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de">systemsafety@lists.techfak.uni-bielefeld.de</a>></div>
<div>Subject: Re: [SystemSafety] Analysis of some Work Breakdown Structure projects</div>
<div><br>
</div>
<div>Steve,</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>Use of a Work Breakdown Structure (WBS) is simply a good project management practice. It’s orthogonal to Waterfall (or other “plan-based”) lifecycles vs. Agile ones.</div>
</blockquote>
<div><br>
</div>
<div>I was surprised to see how many non-software projects use it.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>The WBS is no more and no less than simply a catalog of the work (to be) done on that project...</div>
<div></div>
<div>A running battle I have with the agilista crowd is their insistence that, “our product backlog is our requirements”. No it’s not. Your product backlog is the Agile manifestation of your WBS. The requirements live elsewhere, but that’s another story for
another time.</div>
</blockquote>
<div><br>
</div>
<div>Everybody has their own best practices. What they all</div>
<div>share is a lack of evidence showing almost anything other</div>
<div>than the fact that they are used.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>“I'm not sure of the kind of questions that those involved might ask about such data. Suggestions welcome.”</div>
</blockquote>
<div><br>
</div>
<div>There is an implicit "Assuming the data has the information needed to</div>
<div>answer the question."</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>To me, an extremely useful thing to look at would be “Rework Percentage” (R%). Add up total labor spent repairing defects (it’s a
</div>
</blockquote>
<div>The WBS structure in the data is essentially parent links,</div>
<div>which enables things like figure 2 to be created.</div>
<div><br>
</div>
<div>The rework task information is the same as all the other task</div>
<div>information, e.g., date, time, person-id, etc. But there is</div>
<div>no field specifying what previous tasks have been reworked.</div>
<div><br>
</div>
<div>I was hoping to be able to spot the iterations, but could not</div>
<div>see a simple way to do it. Somebody familiar with using WBS</div>
<div>may be able to see some tell-tale patterns that indicate iterations.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>defect if it is a shortcoming found after the creator of that work claimed that that work was complete, e.g., going into a review or inspection of some sort) and divide that by the total labor on that project.</div>
</blockquote>
<div><br>
</div>
<div>The most common task in the data are inspections of one sort or</div>
<div>another.</div>
<div><br>
</div>
<div>Is reworking a short coming, or an intrinsic part of the process</div>
<div>of building something new?</div>
<div><br>
</div>
<div>Writing software is a process of discovery, and accepting reworking</div>
<div>may be the most cost effect way of progressing.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>The limited but reliable data I have from 5 organizations of between 50 and 350 developers shows a weighted average of 61.2%. In other words, 61.2 out of every 100 labor hours spent in those organizations goes to fixing defects discovered after the worker
claimed their work was complete. This leads directly to the obnoxious but unavoidable conclusion that,</div>
<div></div>
<div>“Rework is not only the single largest driver of cost and schedule on a typical software project; it is bigger than all other drivers combined!”</div>
</blockquote>
<div><br>
</div>
<div>And what approach would be less costly? Those who sing the benefits</div>
<div>of complete upfront design are guilty of hand-waving, just as</div>
<div>much as everybody else.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>Having said that, R% is highly manageable. I don’t have published, reference-able data but I do have experience on software projects that makes me highly confident R% can be well under 10%. I also have anecdotal data that suggests “Acceptance Test Driven
Development” can cut R% in half. R% is the net effect of what Martin Thomas is looking for in timing differences between defect injection and detection. The bigger the difference, the higher R%.</div>
</blockquote>
<div><br>
</div>
<div>Why is 10% better than 90%? It depends on the nature of</div>
<div>the problem.</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>I saw in your paper that the CESAW data set does include rework data. So I think looking at R% both overall and on a project-by-project basis would be interesting and useful.</div>
</blockquote>
<div><br>
</div>
<div>There is lots of interesting data in there.</div>
<div>'All' that is are good ideas to connect it all</div>
<div>together :-)</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div></div>
<div>— steve</div>
<div></div>
<div></div>
<div></div>
<div></div>
<div>-----Original Message-----</div>
<div>From: systemsafety <<a href="mailto:systemsafety-bounces@lists.techfak.uni-bielefeld.de">systemsafety-bounces@lists.techfak.uni-bielefeld.de</a><<a href="mailto:systemsafety-bounces@lists.techfak.uni-bielefeld.de>">mailto:systemsafety-bounces@lists.techfak.uni-bielefeld.de></a>>
on behalf of Derek M Jones <<a href="mailto:derek@knosof.co.uk">derek@knosof.co.uk</a><<a href="mailto:derek@knosof.co.uk>">mailto:derek@knosof.co.uk></a>></div>
<div>Organization: Knowledge Software, Ltd</div>
<div>Date: Tuesday, June 8, 2021 at 3:07 AM</div>
<div>To: "<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de">systemsafety@lists.techfak.uni-bielefeld.de</a><<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de>">mailto:systemsafety@lists.techfak.uni-bielefeld.de></a>" <<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de">systemsafety@lists.techfak.uni-bielefeld.de</a><<a href="mailto:systemsafety@lists.techfak.uni-bielefeld.de>">mailto:systemsafety@lists.techfak.uni-bielefeld.de></a>></div>
<div>Subject: [SystemSafety] Analysis of some Work Breakdown Structure projects</div>
<div></div>
<div>All,</div>
<div></div>
<div>Until recently data on the use of Agile in safety critical software</div>
<div>was almost non-existent.</div>
<div></div>
<div>I'm not sure if the use of work breakdown structure counts as</div>
<div>Agile, but here is an analysis of lots of data (project 615</div>
<div>was one of the safety critical projects):</div>
<div><a href="http://arxiv.org/abs/2106.03679">http://arxiv.org/abs/2106.03679</a></div>
<div></div>
<div>Never having used WBS, I'm not sure of the kind of questions</div>
<div>that those involved might ask about such data. Suggestions</div>
<div>welcome.</div>
<div></div>
<div>--</div>
<div>Derek M. Jones Evidence-based software engineering</div>
<div>tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com</div>
<div>_______________________________________________</div>
<div>The System Safety Mailing List</div>
<div><a href="mailto:systemsafety@TechFak.Uni-Bielefeld.DE">systemsafety@TechFak.Uni-Bielefeld.DE</a><<a href="mailto:systemsafety@TechFak.Uni-Bielefeld.DE">mailto:systemsafety@TechFak.Uni-Bielefeld.DE</a>></div>
<div>Manage your subscription: <a href="https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety">
https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety</a></div>
<div></div>
<div></div>
</blockquote>
<div><br>
</div>
<div>-- </div>
<div>Derek M. Jones Evidence-based software engineering</div>
<div>tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com</div>
<div><br>
</div>
</body>
</html>