[SystemSafety] The Rush

Fri Dec 9 02:17:41 CET 2016

Hi 

There are many things that can spook a herd of bullocks. A possum, a
cracking twig, a flash of lightning or a bush rat chewing on a tender hoof.
One minute the herd is at peace under the stars and the next the're up and
rushing. The Americans call this a stampede, in Australia it's a rush. All
drovers have seen this many times and they know how to shut one down. They
have to before it rolls over the camp site and kills sleeping men in their
swags. Money is at stake too. You can lose cattle in a bad rush and a drover
gets paid on how many bullocks he delivers at the other end , sometimes more
than 2000 kilometres down the track.

When a rush happens a drover is often alone. One man circling a camped herd
of 1500 on a dark night watching for strays while his mates get some sleep.
Here's the thing. A rushing herd always moves in one direction with the fast
movers (thought leaders) out front and the slower followers bringing up the
rear. The trick is to get to the leaders and turn them in a circle back on
the herd so you end up with a heaving mass of sirloin going round and round
and round. Even the most primitive beasts can see the futility of this so
they ultimately calm down and the rush is over. It sounds simple but it's
not. You'd better be a skilled stockman and, if it's night, be on a good
'night horse', one with good night vision and surefootedness in the dark.
These horses are prized among drovers, they save lives.

I heard all this from an old drover interviewed on Australia's ABC radio
national and I couldn't resist the metaphor. It so accurately describes the
state of technology today and hints at what we should do. 

Technology today is in a rush. Spooked by the odd possum (read: imagined pot
of gold) various risktakers are up and bolting to nowhere in particular
tearing through the scrub more focused on the act of galloping (read: I'm
doing it because I can) rather than the destination or the consequences. And
in the process much of what we've learnt in the past 50 years about making
systems safe and secure is being ignored either through ignorance or greed
or man's unquenchable desire to slaughter his enemies, be they logical or
physical. This became clear at the IET's London conference on safety and
security this October. Where I was shocked to learn of the very basic
security measures that ARE NOT being taken by many organisations deploying
web apps in critical applications such as electricity infrastructure. For
example when I asked if SCADA products with Internet connectivity are
delivered standard with firewalls the presenters laughed. Ha, ha. Then we
spent an hour listening to the disembodied voice (via Skype) of a Ukrainian
cyber warrior detailing how Russian hackers shut down a Ukrainian power
network turning out the lights on 250,000 people. It seems the Internet has
been a war zone for some time now.

So where are the stockman on their night horses at the head of the rush
turning the wrong thinking thought leaders back into the pack where we hope
rationalism will prevail. At the conference, UK government representatives
talked of 'kinetics'. "We will no longer just defend. We plan to attack the
source of hacks," they said. Naming and shaming was also popular. Then there
are the regulators. I asked one of them what they plan to do about Tesla's
fraudulent proven-in-use claims of reliability and safety on a vehicle that
has regular software upgrades. He referred me to a website. "Fill out the
form," he said. He reminds me of a drover who climbs a tree when a rush is
on.

So what should we do? My thought is that all professionals have their
surefooted night horses. The courses of action that we know work (and the
ones that we know will certainly end in tears). All we have to do is trust
in them just as a galloping stockman trusts the animal beneath him in the
pitch black amongst a herd of panicked 1000 kg bullocks (picture Elon Musk
at full stretch). And with that certainty we head for the leaders of the
rush and aggressively turn them. That is, aggressively attack bad ideas
where ever they occur.

For example take the statement:

"Suppose you have some software whose performance you have assessed through
operational experience and testing. You know to 95% confidence that the
software has a failure likelihood of less than 1 in 10^(-6) per operational
hour."

My night horse, the one I've been riding for 40 years, tells me that this
statement has a subtext, a way of thinking, that needs to be turned: the
notion that you can derive a numerical indicator of failure likelihood on
software as a function of experience and testing.

This is my belief because:

1. Every single body of software I have dealt with for the last 40 years
has, throughout its operational life, been constantly subject to upgrade.
Changing one line of code invalidates any previous confidence gained from
operational experience (this seems to be lost on Tesla's marketing
department). My worst scare was a one LOC defect that nearly destroyed a
chemical reactor.

2. Testing does not confirm the absence of all defects.

3. Regression testing, post-maintenance, is universally poorly done. There
will always be a conflict of interest between quality and money.

4. Given that most complex systems these days run into millions of SLOCs and
use a variety of software libraries over which the developer has no control,
paranoia is a better state of mind than confidence.

Depending on the above notion to claim software safety is like giving a
bulloch an IQ test to claim it can make it to the rail head 2000 kilometres
away without a drover. Which brings me to my point. If confidence can be
attained at all it is by examining the drovers who manage the herd together
with their excellent night horses. Back in the day this was my understanding
of the thrust of 61508 - "... if the processes are best practice and the
people are highly skilled we will allow you to use the software in critical
applications. ..." I hope this has not changed.

And as for the rest of ya, wake up, get out of your swags, the cognitive
rush is on in a new silly season of software development, I'm nearing the
leader, bolting because he can, admiring the mathematical brilliance of his
predictions on the behaviour of the 10 line program when the rest of us
cattlemen have to find our way in the dark to deal with a herd of millions.
I'm turning, I'm turning him. Help me, you will be rewarded as Banjo
Paterson said:

And the bush hath friends to meet him, and their kindly voices greet him

In the murmur of the breezes and the river on its bars,

And he sees the vision splendid of the sunlit plains extended,

And at night the wond'rous glory of the everlasting stars. 

(From Clancy of the Overflow)

Merry Christmas all

Les

-------------------------------------------------
Les Chambers
Director
Chambers & Associates Pty Ltd
 <http://www.chambers.com.au> www.chambers.com.au

Blog:  <http://www.systemsengineeringblog.com/>
www.systemsengineeringblog.com

Twitter:  <http://www.twitter.com/chambersles> @ChambersLes
M: 0412 648 992
Intl M: +61 412 648 992
Ph: +61 7 3870 4199
Fax: +61 7 3870 4220
 <mailto:les at chambers.com.au> les at chambers.com.au
-------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20161209/076c655d/attachment.html>