[SystemSafety] On open-sourcing coding standards

[Paul Sherwood]

On 2018-09-18 20:27, Olwen Morgan wrote:
> I have some sympathy with the idea that coding standards ought to be
> part of the open-source world but I can see some practical problems
> with the idea (as well as problems with coding standards that are not
> open-source).

Well, I'm also of that view, and I agree that there are problems, but in 
effect you're already demonstrating a way to solve them... this is 
discussion on a public list, which I think provides an excellent basis 
for advancement.

> Here goes:
> 
> I've already spoken of document N0228 on the ISO/IEC-JTC1/SC22/WG23
> web site, which I prepared while participating in the work of the BSI
> Panel on language vulnerabilities. One of the reasons I allowed that
> document to go forward as a Canadian contribution is that it coincided
> more or less with the release of the CERT C coding standard by Robert
> Seacord and his colleagues. I don't very much like the CERT standards
> that I've seen because they do not separate the issues of precise
> identification of language constructs and the controls that need to be
> applied to them. My N0228 document worked through all of the C99
> standard, under the same headings as the standard, identifying where
> particular language constructs needed to be identified and using the
> SYMELAR syntactic metanotation to specify them.

I confess I can't process the technical details at the level you need 
for adequate feedback on this paragraph.

However, I'm a small-scale "mover-shaker" who happens to have a bit of 
standing in a few open source communities. Perhaps more importantly I 
have some good personal relationships with others who have much more 
standing, and much more ability to grasp the nuances of the topics you 
are commenting on. I'll attempt to get relevant folks to read and 
comment.

> Developers of commercial coding standard checkers have an incentive
> not to want coding standards to be written in this way because it is
> not hard to write a checker that is driven directly off the
> metanotation.

Yup. once it's done that way, checker-vendors may struggle to 
differentiate, and thus fail to make a profit.

> Indeed, I strongly suspect that certain vested
> commercial interests (no names, no pack drill - but you could easily
> find out who I mean) made sure that MISRA C 2004 made only minimal use
> of the metanotation. Open-source development of an adaptable
> multi-purpose C coding standard ought, free from vested commercial
> interests, to escape such pressures.

I'm with you, and I'm sure that there are multiple communities that 
would freely take up the challenge if properly advised.

> If you look at N0228,  and compare it with the MISRA C documents, you
> should be able to see fairly easily which constructs are the subject
> of MISRA C rules and how, therefore, to construct a coding standard
> technically equivalent to MISRA C (whichever version you choose)
> simply by constructing a table showing what controls apply to which
> identified constructs. Indeed if the developers of the latest MISRA C
> standard had done this, they could have made the work of producing a
> standard considerably easier for themselves - albeit at the expense of
> p!ssing off the said vested interests.

No comment necessary - I think you've covered it admirably :-)

> The C11 standard runs to 700+ pages. Doing for C11 what I did for C99
> is therefore not a small exercise. On the other hand, nor would it be
> a particularly difficult thing to do. Indeed I have been wondering of
> late whether I ought to make use of my current semi-retired status to
> do just that. That said, however, there are some possible
> non-commercial objections to going that way. One, which was raised
> early in the development of MISRA C 2004, is that the user
> constituency for the standard "would find the syntactic metanotation
> too unfamiliar". My response to that is that SYMELAR is simply a
> slightly augmented form of BNF, and if a software engineer does not
> understand BNF, he/she shouldn't be working on critical systems (and a
> significant number thereof should, IMO, be taken out and shot).
> Nevertheless, acceptability to users cannot be wholly ignored, if only
> to avoid treading on political landmines.

OK, here's the nub of it.

I previously retired (young), then went a bit crazy for a while, and 
returned after about a decade, humbled... to learn what I could about a 
topic in which I had previously already considered myself expert.

 From my vantage point now, I would just argue that safety is something 
that we should all seek, as humans, irrespective of which commercial (or 
other vested) interests we piss off in the process.

As you've probably observed already, I'm looking into autonomous 
vehicles. If we get that particular endeavour wrong, lots of people, 
including my children and their loved ones, may be harmed.

So I'm not particularly interested in the political or commercial 
boundaries. If others create a dangerous solution without proper input, 
I and people I care about are at risk. It's a society-level topic, not 
just a commercial advantage play.

I am expressly interested in establishing and documenting genuine 
good/better/best practice in public, and then doing whatever it takes to 
encourage (or shame, if necessary) practitioners into applying it.

> Another more substantial argument is that given a set of coding rules,
> the developer of a checker tool has a wide choice of how to enforce
> them. There is, for example, quite a difference between, say, QAC and
> the corresponding LDRA checker in what constructs their checkers
> actually look for. Maldadroit choice of designated constructs could
> easily and unfairly favour one existing tool over another. (Having
> been most familiar with QAC, I got some pointed comments from various
> people from LDRA when I gave SYMELAR definitions as input to the early
> stages of developing MISRA C 2004.) This problem, however, can be
> addressed by looking at the messages that individual tools produce. I
> had access to the entire repertoire of QAC diagnostic messages but not
> to those of the LDRA tools, which was understandable in the
> circumstances but nonetheless unhelpful.

As you can see from my comments above, I think the opportunity is in 
rising above the specific companies/organisations that are justifiably 
interested in their own commercial position. I'm sure I'm not alone in 
that :-)

> As it happens I am currently working my way through the C11 standard
> in connection with one of my current back-burner projects. But, by
> hook or by crook, I could probably get the current C11 document into
> MS Word and then do another N0228 style document that I would be happy
> to put into an open-source coding standard project. I expect, however,
> that this would elicit vigorous objections from the current MISRA C
> working group, not to mention various (IMO incompetent) sources within
> ISO/IEC-JTC1/SC22/WG14, where there seems to be no culture sympathetic
> to - or even technically aware of - the needs of critical systems
> development.

 From my perspective I wuld say, just ignore the objections. That may not 
be a viable strategy for others in general.

> Having seen how MISRA C has gone since 2002, I am singularly
> unimpressed by the results and have, I am afraid, very little
> confidence in the ability of that working group collectively to make
> the kind of technical progress with the standard that would make it
> robust enough to be acceptable for SIL3/4 work (which it is nowhere
> near right now). As regards the C standard, my confidence in WG14 is
> not describable in polite terms. These days I regard the processes of
> international and industry-sector standardisation as dysfunctional and
> currently incapable of producing either languages or coding standards
> that are truly suited to the highest levels of system criticality
> (SPARK Ada being a lone commendable exception).

I'm with you on the standards. On this list I've already called the IEC 
charging model a disgrace. Elsewhere in public I've seen the standards 
ecosystem described as a Ponzi scheme (and I think I have to agree that 
it's not much better than that).

My weakness in all of this though is that I can't actually digest all of 
the technical details... I'm a so-so C programmer at best. Luckily I do 
know some actual experts ;-)

> On a more positive note, I am very impressed with French work in this
> area on analysis tools, notably Frama C and AbsInt - the
> commercialised version of Astree. If the developers of those tools
> have pressed ahead because they have given up on standardisation
> dominated by a single, delapidated, national technical culture, then I
> wish them a huge "bonne chance" (which, coming from a deeply
> anglo-sceptic Welshwoman, is about what you should expect).
> 
> Tin hats on and head to the nuclear bunkers,

Before we burrow underground, maybe another tilt at the windmills?

best wishes
Paul