[SystemSafety] OpenSSL Bug

Wed Apr 16 17:47:27 CEST 2014

"
And even if there were no explicitly stated requirement, had the developer had any brains and used Design-by-Contract then they would surely have noticed that the value range of that parameter's data type was much bigger than what the function was supposed to be able to handle. The need for a range check would have been bloody obvious.
"
There are two things here.

Is the programmer expected to have a brain, and if yes of which type (requirement developer ?) ?

Is a programmer expected to develop requirements when thinks they are missing ?

Isn't a healthy development process (and this is not restricted to SW) based on sticking to the requirements and eventually going back to requirement capture rather than engineering requirements in lower phases of the development cycle ?

Bertrand Ricque
Program Manager
Optronics and Defence Division
Sights Program
Mob : +33 6 87 47 84 64
Tel : +33 1 58 11 96 82
Bertrand.ricque at sagem.com

-----Original Message-----
From: systemsafety-bounces at lists.techfak.uni-bielefeld.de [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Steve Tockey
Sent: Wednesday, April 16, 2014 5:36 PM
To: David MENTRE; systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] OpenSSL Bug

David,

"Probably because the whole discussion started from a bug (in OpenSSL) that precisely lies in those 7%. :-)"

Actually I'm going to respectfully disagree with that. I'm going to claim I can trace it back to incomplete requirements & resulting inadequate design. A parameter on a function call was missing a range check. Why?
Because there wasn't an explicit requirement that the value represented in that parameter be within a specific range. Had there been an explicit requirement, and had the designer/programmer paid attention to that requirement then they would have needed a range checking 'if' statement to satisfy that requirement.

And even if there were no explicitly stated requirement, had the developer had any brains and used Design-by-Contract then they would surely have noticed that the value range of that parameter's data type was much bigger than what the function was supposed to be able to handle. The need for a range check would have been bloody obvious.

It wasn't at all a 7% defect, it was an 83% defect.

"More seriously, I would make the distinction between two kinds of
approaches: ..."

Again, slight disagreement. I don't know the exact source of the defect injection data (I.e., what specific kinds of projects the data came from).
However, I do know (of) the person who published the data: James Martin.
Knowing (of) Mr. Martin, I'm hard pressed to say that the data came from any safety critical projects.

Besides that, my own experience with both safety-critical and non- development projects doesn't show any appreciable difference in the defect
*injection* statistics. The big difference, IMHO, is that the safety-critical projects make much better use of reviews and the defect
*detection* statistics are heavily skewed towards being found and fixed earlier.

"Thus I have a practical question: suppose I have a company that lacks such good practices (at least for the review or safe coding rules one).
I'm not a project manager, just a low level software engineer working on a given project. How can I help improve the development process to reach such level of good practices?"

Contact me directly, I'll send you a paper called "How Healthy is your Software Process?". By instrumenting the project's defect tracking system with very simple data we can easily show that the majority of defects are requirements & design injected and that more than half of the organization's capacity to do development work is being wasted in finding and fixing mistakes that were made much earlier. Managers have a tendency to discount what the techies are telling them, but it's hard to ignore the data when it's staring them in the face.

If anyone wants the paper, I'm happy to share it.

-- steve

-----Original Message-----
From: David MENTRE <dmentre at linux-france.org>
Date: Wednesday, April 16, 2014 6:32 AM
To: "systemsafety at lists.techfak.uni-bielefeld.de"
<systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] OpenSSL Bug

Hello,

Le 16/04/2014 13:23, Steve Tockey a écrit :
> No offense inferred, but thanks for pointing it out.

The same for me. I should have kept the classical "if (a == SUCCESS)"
example. Another example: C language does not forces you to check the return value of a function (that might flag an error case). Most of other languages I know would force you to check it (or at least issue a warning).

> Besides, echoing this and Derek M. Jones' last post, I think this
> whole discussion is focusing on relatively minor issues and completely
> missing the big picture.
[...]
> the simple fact is that 83% of
> all defects exist before a single line of code was ever written. Why
> aren't we attacking the 83%, not the 7%???

Probably because the whole discussion started from a bug (in OpenSSL) that precisely lies in those 7%. :-)

More seriously, I would make the distinction between two kinds of
approaches:

  1. People that produce software following the good practices you describe below, probably to satisfy DO-178C or EN 50128 standards. In such case, their error figures are probably in the above range (83% requirement + design, 7% code).

  2. People that produce software *without* following proper practices, probably like OpenSSL developers and most companies developing software in non-safety critical domain. In that case the coding errors are much more prevalent. And we are several people on this list to think that using better tool could help catch more of these errors. I do agree that following good practices could have similar effect, but the simple fact is that they are not doing it, probably due to lack of knowledge, time, money or because this is not a paid-for job. You might counter-argue that if they are not using good practices, they won't use static analysis tool neither. You might be right. ;-)

> A
> team with good requirements, design, and review practices can write
> good, safe code in any language, including C.

I agree.

Thus I have a practical question: suppose I have a company that lacks such good practices (at least for the review or safe coding rules one).
I'm not a project manager, just a low level software engineer working on a given project. How can I help improve the development process to reach such level of good practices?

Sincerely yours,
david

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
#
" Ce courriel et les documents qui lui sont joints peuvent contenir des informations confidentielles, être soumis aux règlementations relatives au contrôle des exportations ou ayant un caractère privé. S'ils ne vous sont pas destinés, nous vous signalons qu'il est strictement interdit de les divulguer, de les reproduire ou d'en utiliser de quelque manière que ce soit le contenu. Toute exportation ou réexportation non autorisée est interdite Si ce message vous a été transmis par erreur, merci d'en informer l'expéditeur et de supprimer immédiatement de votre système informatique ce courriel ainsi que tous les documents qui y sont attachés."
******
" This e-mail and any attached documents may contain confidential or proprietary information and may be subject to export control laws and regulations. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. Unauthorized export or re-export is prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."
#