<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div><br>
</div>
Dewi,
<div>Thank you very much for the clarification. I appreciate it. I agree 100% with you.</div>
<div><br>
</div>
<div><br>
</div>
<div>Best,</div>
<div><br>
</div>
<div>— steve</div>
<div><br>
</div>
<div><br id="lineBreakAtBeginningOfMessage">
<div><br>
<div>On Aug 8, 2024, at 1:31 AM, Dewi Daniels <dewi.daniels@software-safety.com> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div dir="ltr">
<div dir="ltr">On Wed, 7 Aug 2024 at 19:03, Steve Tockey <<a href="mailto:steve.tockey@construx.com">steve.tockey@construx.com</a>> wrote:<br>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div><br>
</div>
<div>Paul,</div>
<div>I should have jumped on this from the start …</div>
<div><br>
</div>
<div><i>“DO-178C doesn't require you to achieve 100% structural coverage</i></div>
<div><i>through unit tests. It requires that your requirements-based tests</i></div>
<div><i>(which can be hardware/software integration tests, software</i></div>
<div><i>integration tests or low-level tests) cover all the software</i></div>
<div><i>requirements and exercise all the code.”</i></div>
<div><br>
</div>
<div>That’s not at all how I read DO-178C. Yes, DO-178C does mandate what I would call “100% Requirements Coverage”. But in addition, Table A-7 on Structural Coverage Analysis clearly does mandate (quote):</div>
<div><br>
</div>
<div>Row 5<span style="white-space:pre-wrap"> </span>"Test coverage of software structure (modified condition/decision) is achieved” for DAL A</div>
<div>Row 6<span style="white-space:pre-wrap"> </span>“Test coverage of software structure (decision coverage) is achieved” for DAL A , B</div>
<div>Row 7<span style="white-space:pre-wrap"> </span>“Test coverage of software structure (statement coverage) is achieved” for DAL A, B, C</div>
<div><br>
</div>
<div>While DO-178C may not explicitly mandate that said Structural Coverage be demonstrated at the unit test level, it is often impossible in practice to achieve it in any other way.</div>
</div>
</blockquote>
<div><br>
</div>
<div>What I wrote is correct. I agree that DO-178C requires test coverage of software structure at Levels A, B and C. However, this can be achieved through a combination of hardware/software integration tests, software integration tests and low-level tests
as shown in DO-178C figure 6-1. I also agree that achieving structural coverage will invariably require at least some unit tests. This is acknowledged in the note to DO-178C §6.4.1, which reads "In many cases, the requirements-based coverage and structural
coverage necessary can be achieved only with more precise control and monitoring of the test inputs and code execution than generally possible in a fully integrated environment. Such testing may need to be performed on a small software component that is functionally
isolated from other software components".</div>
<div><br>
</div>
<div>I was responding to Paul's response to my question</div>
<div><br>
</div>
<div><span style="color:rgb(80,0,80)">> Even Level C isn't that hard. Why wouldn't</span><br style="color:rgb(80,0,80)">
<span style="color:rgb(80,0,80)">> you want to achieve statement coverage?</span><br style="color:rgb(80,0,80)">
</div>
<div><br>
</div>
<div>in which he wrote</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Actually, I wouldn't, because of my personal bias. I've successfully<br>
delivered small amounts of critical code (in the few thousands of LoC<br>
range) without unit tests, for systems which then worked for years<br>
without problems. Usually I've had more success with system tests than<br>
unit tests, on many projects.<br>
</blockquote>
<div><br style="color:rgb(80,0,80)">
</div>
</div>
<div>Paul seemed to be assuming that statement coverage is achieved through unit testing alone. That was my point in writing "DO-178C doesn't require you to achieve 100% structural coverage through unit tests". I should have added "alone".</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>In fact, “Statement Coverage” as a Structural Coverage criterion is the absolute weakest of all Control-Flow based Structural Coverage criteria and it has logical holes in it so big that one could drive a proverbial double decker London City bus through
them.</div>
</div>
</blockquote>
<div><br>
</div>
<div>Absolutely. My original comment was "I don't understand why it's so hard for Linux to just comply with standards such as RTCA DO-178C/EUROCAE ED-12C? DO-178C Level D is just Software Engineering 101. Even Level C isn't that hard. Why wouldn't you want
to achieve statement coverage?". I find it very hard to understand why it's so difficult to create a 'Safe Linux' that complies with DO-178C Level C, let alone DO-178C Level D. As Prof. Peter Ladkin wrote, "This business about Linux kernel for safety-related
systems has been going on for so long. Other companies have written kernel-function OSs for safety-critical systems, and have assessment certificates for them from recognised assessors, all within that time. What's wrong with trying that route?".</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>And the vast majority of organizations I work with only aim for 60% to 70% Statement Coverage for even their most critical code. Shocking.</div>
</div>
</blockquote>
<div><br>
</div>
<div>There's a very big gap between safety-critical software and other kinds of software. There always has been. I don't entirely understand why. </div>
<div><br>
</div>
<div>Yours,</div>
<div>Dewi </div>
</div>
</div>
</div>
</div>
<br>
</div>
</body>
</html>