[SystemSafety] OpenSSL Bug

Derek M Jones derek at knosof.co.uk
Mon Apr 14 16:48:02 CEST 2014 

Nancy,

> As you give no references, I don't know what papers you are talking about.
> But the ones by John Gannon and other respected researchers had appropriate
> experimental design.  It is always easy to criticize the results of any
> experiment. But do you have any experimental results you can cite that show
> that these features are not error prone?

Gannon's paper: "An Experimental Evaluation of Data Type Conventions"
describes what looks like a controlled experiment and the results
look good.

Two different languages were used NT and ST, which presumably differed
in all sorts of ways.  Why is all the credit for the difference seen
assigned to strong typing?

The metric used for comparison was total number of errors.  This is
a rather broad measure: what is the impact of compiler implementation
differences (both compilers tarted from a common base, which has the
potential to reduce this impact), were some subjects more likely to
make minor mistakes in one language and not the other?

This is an interesting experiment that has an interesting result.
It needs to be replicated and refined before anything concrete can
be claimed with any confidence.

It has a depressingly small number of citations, 93, for such a good
paper.

> Nancy
>
>
> On Mon, Apr 14, 2014 at 10:15 AM, Derek M Jones <derek at knosof.co.uk> wrote:
>
>> Nancy,
>>
>>
>>   Actually, there is a lot of scientific evidence (better than empirical,
>>> although there is a lot of empirical evidence too). There were a lot of
>>> studies done in the 1980s showing error-proneness of particular
>>> programming
>>> constructs. The non-typed language features were the most error-prone.
>>> John
>>> Gannon did some of them.
>>>
>>
>> I have seen a fare few of these and they all suffer from poor
>> experimental design and small sample size.  The searched for effect
>> is given all the credit when a result goes in the desired direction
>> without any analysis of what other factors might have made this
>> happen.
>>
>>
>>   More recently, there have been studies comparing SPARK and non-strongly
>>> typed languages. Martyn Thomas should have more information about that.
>>>
>>
>> There is this paper (thanks to Dewi Daniels for the link):
>> http://www.crosstalkonline.org/storage/issue-archives/
>> 2003/200311/200311-German.pdf
>>
>> But not a lot of rigor there.
>>
>> I have been other papers that do more analysis, but the conclusion that
>> it is all down to strong typing gets pulled out of a hat.
>>
>>
>>   I've also seen several papers on comparisons from industry, not student
>>> programmers. I don't have time to look them up, but I've assigned them to
>>>
>>
>> The following is also a reply to Paul D. Stachour's email.
>>
>> There are obvious advantages to using strong typing on projects that
>> start from scratch and don't make use of lots of third party libraries.
>>
>> Assuming that the developers make use of the facilities provided by
>> strong typing (its surprising how many think it is an inconvenience
>> and actively try to get around it) then:
>>
>> In any language there are invariably multiple ways of using the strong
>> typing functionality.  So in a large team there is the potential for
>> interface problems caused by different approaches to strong typing (this
>> also occurs when third party libraries are used).
>>
>> Of course developers claim there is the one true way to do things.  If
>> different developers would agree on the one true way the problem
>> disappears.
>>
>> There is also the issue of what to do about runtime checks.  Code
>> is surprisingly robust in the presence of minor infringements, it
>> works as intended.  But if runtime checking is on an error gets raise.
>> Having an error raised can cause more problems than ignoring the
>> problem (the safety people on this list will know a lot more about this
>> issue than me).
>>
>> So taken in totality it is not clear to me that strong typing is
>> actually more cost effective on larger projects or projects
>> where lots of third party code is used (which is now the common
>> case for most projects).
>>
>>
>>
>>   my classes in the past. I think that not much is done on this topic by
>>> academics and researchers anymore because there doesn't seem to be any
>>> doubt about it.
>>>
>>> Nancy
>>>
>>> careful
>>>
>>>
>>> On Thu, Apr 10, 2014 at 3:06 PM, Derek M Jones <derek at knosof.co.uk>
>>> wrote:
>>>
>>>   Peter,
>>>>
>>>>
>>>>    There are people here who have defended the use of the programming
>>>>
>>>>> language C. Shame on you. Yes,
>>>>>
>>>>>
>>>> Why pick on C?  All language have their problems.
>>>>
>>>> Facebook have been doing good stuff to improve the reliability of PHP:
>>>> http://shape-of-code.coding-guidelines.com/2014/03/24/
>>>> hack-a-template-for-improving-code-reliability/
>>>>
>>>>
>>>>    there are tools; there are reliable tools to check whether C programs
>>>>
>>>>> adhere to strong-typing
>>>>>
>>>>>
>>>> There is no discontinuity that distinguishes weak/strong typing, it is
>>>> a continuum.  Good luck reaching general agreement on where to draw
>>>> the line.
>>>>
>>>> I have worked in languages that have stronger typing than C and
>>>> seen plenty of code in those languages where developers have failed
>>>> to use the strong typing facilities available to them.  Giving
>>>> developers the tools does not mean they will use them (I am a fan
>>>> of stronger typing than is available in C).
>>>>
>>>> Incidentally there is almost no empirical evidence for the benefits
>>>> of using a language having stronger typing.  There are a few studies
>>>> using students on really small problems.  Pointers to good studies
>>>> welcome.
>>>>
>>>>
>>>>    principles. Etc. AND THEY WERE NOT USED BY PEOPLE WHOM I HAVE UP TO NOW
>>>>
>>>>> TRUSTED. In other words, you
>>>>> were lying to us about "good practice" amongst "SW developers" using C.
>>>>>
>>>>>
>>>> and you are surprised by this (again why pick on just C)?
>>>>
>>>> --
>>>> Derek M. Jones                  tel: +44 (0) 1252 520 667
>>>> Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
>>>> Software analysis               http://www.knosof.co.uk
>>>>
>>>> _______________________________________________
>>>> The System Safety Mailing List
>>>> systemsafety at TechFak.Uni-Bielefeld.DE
>>>>
>>>>
>>>
>>>
>>>
>> --
>> Derek M. Jones                  tel: +44 (0) 1252 520 667
>> Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
>> Software analysis               http://www.knosof.co.uk
>> _______________________________________________
>> The System Safety Mailing List
>> systemsafety at TechFak.Uni-Bielefeld.DE
>>
>
>
>

-- 
Derek M. Jones                  tel: +44 (0) 1252 520 667
Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
Software analysis               http://www.knosof.co.uk

More information about the systemsafety mailing list