[SystemSafety] "Ripple20 vulnerabilities will haunt the IoT landscape for years to come"

[Martyn Thomas]

My take on the unit test question is this. Software Engineering is a
practical activity. It is claimed that Neville Shute said "an engineer
is a man who can do for ten shillings what any fool can do for a pound".
I wouldn't go that far - some engineers can do things that no fool can
do at all. But I agree that cost-effectiveness is an important property
of a software engineering process.

There are always more and different tests you could run, so there will
always be a need to decide which tests add most confidence or save
significant costs later. The argument that I have heard for omitting
many/most/all unit testing, as part of a SPARK development, is that a
key role of unit testing is to find things that would cost a lot more to
fix if you only found them in integration, system or user testing and,
in practice, a SPARK development wouldn't find anything much at all in
any cost-effective amount of unit testing, so it's a sensible
engineering decision to omit it.

That's an engineering judgement. There may be particular system units
that for one reason or another you feel less confident about and want to
unit test extensively. It's a question of deciding whether the effort
involved would be a good use of available resources.

That's my current opinion. But I'm willing to change it if I'm given a
convincing argument by those who have far more recent experience of such
developments than I have.

Martyn

On 01/07/2020 17:47, Peter Bernard Ladkin wrote:
>
> On 2020-07-01 18:10 , Olwen Morgan wrote:
>> Not so. There is always the possibility that the verification and testing tools contain errors.
> There is also the possibility that when you type a "2" on your keyboard as a test input, "5" is
> actually input, and when you see a value on your screen, the value that is input to the display is
> actually different from what you see. And so on.
>
> Altran UK performs (performed; I don't know whether they still do) a daily build on the iFacts
> system. They don't perform unit tests on all the modified SW elements; they generate proof
> obligations and discharge them in a daily run of the prover. As I said, if you use CbyC you can
> avoid unit tests. If you don't think that is possible, then I suggest you write letters to the DoT
> and NATS and your local MPs to tell them that Altran is pulling the wool over their eyes.
>
> Which you won't do, because you well know that Altran is not doing so, and that iFacts is working
> and continues to work. Without extensive unit tests on modified modules.
>
>> You use SPARK as an example but what about users of C? 
> It may well be that users of C have to unit test until they are blue in the face and their
> fingertips have calluses. That has little to do with the point I made.
>
>> .... I'll gladly use the best verifiers and testing tools I can get my hands on. But
>> I will not depart from viewing testing as an experiment that tries to falsify the correctness
>> hypothesis. Once you adhere to that view, the results of testing always have the possibility of
>> contradicting the results of verification. 
> Every test you run has the same plethora of "what ifs" as any other process which you initiate from
> your keyboard and for which you read the results from your screen. But some of those "what ifs" are
> likelier than others and one puts resources where they are most effective. Driving into town to have
> your kit supplier test that your keyboard is wired properly is not necessarily the best use of
> resources when developing critical programs and I doubt your client would be content to pay for it.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Bielefeld, Germany
> Styelfy Bleibgsnd
> Tel+msg +49 (0)521 880 7319  www.rvs-bi.de
>
>
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20200701/74224724/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20200701/74224724/attachment.sig>