[SystemSafety] Current Affairs in Standardisation
Peter Bernard Ladkin
ladkin at rvs.uni-bielefeld.de
Fri Dec 10 12:09:25 CET 2021
So, nothing much has been going on on this list for a while. Some of this, I think, is down to
changes in the communication environment. Back 20 years ago, email was the way people with a
computer talked to each other. As far as my own habits are concerned, it still is, along with
short-messaging (then, SMS; now, WhatsApp/Signal/Threema).
There is stuff going on in system safety. Let me address electrotechnical standardisation here.
First, the 61508 maintenance teams are nearing agreement on a CD. The official maintenance phase has
not yet been opened, because the IEC has guidelines that such phases should last at most two years.
The SW part (61508-3) has been meeting regularly since December 2014, so it is now on its seventh
year of "preliminaries".
The General+HW part (61508-1/2) has been meeting regularly since November 2017, so is only on its
fourth year of "preliminaries".
For comparison, IEC guidelines are that inquiries as to a need for maintenance are made amongst the
(country) members of a TC or SC every five years. IEC 61508 Ed 2 is dated 2010. The inquiry about
need for maintenance was distributed in early 2017.
What I and colleagues see is that more and more material is being added to a already very lengthy
and very expensive standard. I had proposed back in mid-201x that new material should first be
proposed in a separate document -- a Technical Specification, which has normative force; or a
Technical Report, which is informational; or even a separate international standard. The
proven-in-use criteria for SW, for example, were written in 2013-2015 after an initial proposal by
Germany (we had ourselves worked on the proposal for a year at least). It became IEC TS
61508-3-1:2016. Its provisions have now been largely incorporated into 61508-3 Ed. 3, and 61508-2
has been modified to remove the hook for the anomalies which led to the need for 61508-3-1.
But it could have stayed as a separate document; or become an International Standard itself, and
61508-3 Ed. 3 could have referred to it.
There is some guidance being prepared on Object-Oriented safety-related SW development. This will be
IEC 61508-3-3 and in fact will replace a highly unsatisfactory two pages in Part 7 Annex G.
A replacement for Part 7 Annex D, the statistical evaluation of software operational histories, has
been formulated. The original was 4.5pp; this is somewhat longer (despite our attempts to keep it
short). There has been talk of an extended guidance document, a TR. Whether there is really a need
for such a thing is not clear. There are some novel techniques not dealt with in the Annex D
replacement, but novelty and standards don't go together - the idea of a standard is to elaborate
industrially-mature techniques, rather than novel techniques. And the general experience with
StatEval of safety-related SW is that the numbers one has mostly don't justify the claims one needs
to make (e.g., the observations of Littlewood-Strigini from 1993); no novel techniques are going to
change that. But there is a continuing, valid need. I have colleagues producing simple but
exceptionally reliable equipment (say SW-driven sensors) who have tens of thousands of these in the
field, essentially without a SW-generated fault in, say, two decades. You want a good argument for
using one of these rather than designing a new one. That has to be based on the operational
experience. But exactly what argument can you make and how can you go about it? That question
remains essentially unanswered even with the replacement Annex D.
Cybersecurity has essentially been "outsourced" to the appalling IEC TR 63069, whose committee is
working on a replacement, which they intend to be normative. It is hard to believe that anyone who
knows much about practical cybersecurity worked on that document (I know people who quit the Working
Group early on, when it became clear what was going to be in it). This is the only IEC document I
know which has been replaced in one of the big standardisation nations, namely GB, with a "local"
version, published by the IET with the collaboration of NCSC, within a year of publication. If that
isn't a clear statement of unsatisfactoriness, I don't know what is. My suspicion, and prediction,
is that people reading the 61508 CD in 2022 will not be content with how it deals with cybersecurity.
The guide to what should be done when formal methods are claimed to be used, IEC DTS 61508-3-2,
whose Project Team I convene, is about to come out with its second CD. (It will be sent to the IEC
for distribution next week.) It is, deliberately, a minimal document. Neither I nor the majority of
members of the PT are friends of bloat. Neither can it be a guide to what formal methods there are
and how to use them, for two main reasons. First, that is at most informative, and the current
document is normative. It says there must be certain kinds of documentation (e.g., formal
comparisons of two levels in a relation of refinement). Second, many successful, mature development
paths based on formal methods are commercial or commercially supported (Esterel/SCADE; SPARK/Gnat
Pro), and the IEC proscribes mention of commercial artefacts in standards.
The hot new theme is of course AI, meaning machine learning, meaning DLNNs. There is an IEC working
group on functional safety and AI, which is coming up with a Tech Report 5469. In principal,
everything AIish is dealt with by ISO/IEC JTC1/SC42, but this tech report is not listed amongst
their publications, and I don't see a listing of projects. In any case, ISO/IEC TR 5469 was said to
be, in April, about to be published, but nine months on I don't think the final document has yet
been submitted. It is a "hot" subject; our German WG meets every two weeks for a couple hours. Along
with it, of course, "experts" on AI and functional safety have been appeared out of the woodwork. A
year ago, I recall introducing many of them to the extensive work NASA did in the 1990's and 2000's
with control of actual flying airplanes (an F-15 and an MD-11) with dynamically-leaning DLNNs, and
the by-now ten-year old book by Schumann and Liu which recounts those and other projects. Further
back, my 1987 PhD thesis was in (symbolic) AI and parts of it are still being regularly cited,
according to acks I get from ResearchGate. So I incline to consider myself somewhat experienced with
the technology, although I wouldn't claim any expertise in training DLNNs. The issue here is what
should go in 61508, if anything, about AI. Ideally, it would be in 5469 or a similar document, but
indeed there are novel aspects of safety assessment, some of which I wrote up in an April note which
I still have to format for submission to the SCSC eJournal. I don't necessarily have good feelings
about outcomes in standardisation - I see a few people profiling themselves as functional safety+AI
experts who knew nothing about the technology a couple of years ago. This is unfortunately to be
expected in engineering standardisation -- big companies want to jump on bandwagons as they form,
even when they haven't (yet) worked with the technologies, so company shills get sent as placeholders.
But, back to the big theme. We have 61508-3-1 PiU requirements for SW; 61508-3-2 FM requirements;
61508-3-3 OO guidance, 63069 FS and cybersec; 5369 FS and AI; 61508-3-X StatEval for SW. That is six
themes in functional safety "outsourced" to separate documents. Wouldn't it be better all round to
have, say, 30-50 such documents and a much thinned-down 61508 which refers to them?
PBL
Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
Tel+msg +49 (0)521 880 7319 www.rvs-bi.de
More information about the systemsafety
mailing list