[SystemSafety] Another unbelievable failure (file system overflow)

[RICQUE Bertrand (SAGEM DEFENSE SECURITE)]

There is currently an IEC workgroup on what to do with IEC 61508 and cybersecurity (IEC 62443). The topic is thus not ignored.

Bertrand Ricque
Program Manager
Optronics and Defence Division
Sights Program
Mob : +33 6 87 47 84 64
Tel : +33 1 58 11 96 82
Bertrand.ricque at sagem.com

From: systemsafety-bounces at lists.techfak.uni-bielefeld.de [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Martyn Thomas
Sent: Monday, June 01, 2015 10:35 AM
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Another unbelievable failure (file system overflow)

Les/Steve

Thanks for this. There's little discussion of professional ethics in any forum that I read.

Do you think there's any hope that we might be able to make a small advance in a focused area, such as IEC 61508? The standard isn't fit for purpose, in that it largely ignores cybersecurity issues and does not provide a sound basis for assessing whether safety-critical systems are safe enough for their proposed application. It's also too long, inconsistent, too expensive, and can't be copied/republished for use in teaching, research or professional debate. I see these weaknesses, in the central international standard for the safety of computer-based systems, as an ethical issue. Do you agree?

Regards

Martyn

On 31/05/2015 05:14, Les Chambers wrote:
Steve
Thanks for referencing the code of ethics. It should be brought up more often. Unfortunately, for me, it makes depressing reading. Especially when you come upon paragraphs such as:

3.12. Work to develop software and related documents that respect the privacy of those who will be affected by that software.

Although he has probably never read it, there is a man, who will probably never see his homeland again because he took these sentiments to heart and attempted his own corrective action. And what of the thousands of scientists, engineers and technologists who contributed to the construction of the software, the existence of which, he exposed to the world?

My point is that non-compliance with this code of ethics is massive and almost universal. In fact, any engineer maintaining strict compliance with every paragraph of this code would be unemployable in our modern world.

Reading these paragraphs through the lens of experience I am blown away by their flippancy. From personal experience I can tell you that screwing up the courage to implement even one of these items can be a massive life changing event. This sentence would be lost on a graduate. They're all perfectly reasonable statements of how one should behave. Much like, "Thou shall not kill, thou shall not commit adultery ...".  The issue lies in the moral courage to implement.

There is no quick fix to this problem as we are a decentralised, unorganised and generally fragmented lot. We don't have the luxury of the medical profession that deals with a single organism. We can't simply state and righteously comply with the notion of, "Do no harm." In fact, for us, the opposite is true, many of us work in industries where the primary purpose is to kill other human beings, and with high efficiency (fewer soldiers kill more enemy).

One thing we can do is deal with the problem at its root:

We are graduating incomplete human beings from science and engineering courses. There is insufficient focus on the moral issues surrounding the impact of our machines on humanity. For example, a study of applied philosophy, including ethics, should be a nonnegotiable component of all engineering courses. Not just a final year subject, but a subject for every year with a weekly reflection on the content. Much like the weekly safety meetings I was forced to attend in the chemical processing industry.

I'm sure there will be howls of laughter at this, but, let me tell you it's the only thing that caused me to back a senior manager about five levels above my pay grade into a corner - he could physically not escape me short of punching me out and stepping over my body - and berate him until he promised to properly train his operators in the emergency procedures for a safety critical system.

Popping a few paragraphs up on the web would never have done the trick.

That experience was trivia compared to where we are headed. The massive computing power now available means that our software is beginning to take higher level decisions away from human beings. Some of these decisions are moral ones (refer my previous post on lethal autonomous weapons systems). "Shall I kill all humans associated with this structure, or no?"

At a recent engineering alumni meeting I asked the head of my old engineering Department how much philosophy is taught to undergraduate engineers. He chuckled. "It is available as an elective but less than one percent participate," he said.

I plan to speak to him again soon.

Cheers
Les


#
" Ce courriel et les documents qui lui sont joints peuvent contenir des informations confidentielles, être soumis aux règlementations relatives au contrôle des exportations ou ayant un caractère privé. S'ils ne vous sont pas destinés, nous vous signalons qu'il est strictement interdit de les divulguer, de les reproduire ou d'en utiliser de quelque manière que ce soit le contenu. Toute exportation ou réexportation non autorisée est interdite Si ce message vous a été transmis par erreur, merci d'en informer l'expéditeur et de supprimer immédiatement de votre système informatique ce courriel ainsi que tous les documents qui y sont attachés."
******
" This e-mail and any attached documents may contain confidential or proprietary information and may be subject to export control laws and regulations. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. Unauthorized export or re-export is prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."
#
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150601/ec6d8375/attachment-0001.html>