<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><p class="p1" style="margin: 0px; font-stretch: normal; line-height: normal; -webkit-text-size-adjust: auto;"><span style="font-size: 21.049999237060547px;">Hi All</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">For a hair raising update on the current ‘state of the art’ in self driving cars, Google “You tube CNN tests a ‘full self-driving’ Tesla” and hang onto your seat.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">I’ll leave it to the list to draw its own conclusions on AI’s utility in driving a land vehicle.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Here are mine:</span></p><ol class="ol1" style="-webkit-text-size-adjust: auto;"><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Dangerously immature. AI technology lacks the maturity to be used in any safety critical control loop. It must be banned immediately.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Dangerously unregulated. The reality of the road vehicle in the video being on a Brooklyn street, piloted by a journalist, fitted with a beta test version of safety critical software, with the demonstrated dangerous failure rate represents a regulatory failure reminiscent of the FAA and the 737 Max.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Dangerously fanciful hazard reduction assumptions. The notion that ‘a human in the loop’ can compensate for these failures , rendering the system acceptably safe , is fanciful. It is well known that over dependence on a human to diagnose and rectify complex problems in high stress situations in real time ends in tears. All our profession has learned about avoiding cognitive overload has been ignored. “Keep your hands on the wheel” ?? Spare me. <span class="Apple-converted-space"> </span>In my work on intelligent traffic management <span class="Apple-converted-space"> </span>systems I came to understand how seriously road authorities take the act of shutting down a freeway. They avoid it where possible as drivers lapse into a somnambulistic state at constant traffic flow rates in cruise control with no episodes requiring evasive action. In the time it takes to wake up, detect stationary vehicles ahead and apply the brake a vehicle can travel a substantial distance at freeway speeds. Hence ‘back of queue’ collisions are common. <span class="Apple-converted-space"> </span>Even in this environment, a best case for auto pilots, I don’t don’t relish the thought of driving a car that behaves like a teenage learner driver.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Dangerous ignorance of the functional safety engineering discipline. The motor vehicle industry seems to be holding all the past 40 year’s progress in safety critical systems engineering in utter contempt (the actors developing these systems may have been unaware of it in the first place). Computer science is being deployed in these vehicles directly without the necessary detour through systems engineering (probably because no AI specific safety engineering process exists as yet). The developers are dosed up on texts such as Russell & Norvig, Artificial Intelligence - A Modern Approach; all science and no engineering. For example, Stuart Russell gave last year’s BBC Reith Lecture “Living With Artificial Intelligence” (available on line - and highly recommended. Stuart is a brilliant speaker.) One of his stated problems is, “We don’t know how to specify an AI yet”. Spoken like a true computer scientist (which is what he is). An engineer would have taken the proposition a step further, “… therefore we are unable to validate an AI.”</span></li></ol><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Why is such a hard won and highly effective engineering discipline being disrespected so? I offer some root causes:</span></p><ol class="ol1" style="-webkit-text-size-adjust: auto;"><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">HUBRIS surrounding electric cars in general and the BISC factor in particular.</span></li></ol><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">(BISC: but it’s sooo cool - usually uttered with forearms raised, fists clenched and moving laterally at high frequency). </span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">The systemic failure modes triggered by hubris are:</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">A. JUST DO IT! Under pressure from innovation crazed governments, underfunded and technologically under powered regulators look the other way. <span class="Apple-converted-space"> </span>Transparently unsafe designs are visited on the public.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">B. IGNORANCE! Developers don’t look at all - no one told them they had to. Well understood safety engineering processes routinely used in parallel universes (aviation, rail, chemical processing) are not applied.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Note that within the next 5 - 10 years the planet will move on its axis under the weight of <span class="Apple-converted-space"> </span>the greatest volume of cash flow ever experienced as Apple announces it’s ‘autonomous self-driving vehicle.’ You have a MacBook Pro, an iPad, an Apple Watch; you must need an Apple car. Kevin Lynch, Apple’s current software development manager has no safety critical software engineering credentials. He cut his teeth on Adobe Dreamweaver and the Apple Watch. Ask him for his hazard log. “Say what boy?” </span></p><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span></p><ol class="ol1" style="-webkit-text-size-adjust: auto;"><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">THE TECHNOLOGY TAIL WAGS THE PROCESS DOG. Tesla is a case study of an attempt to fit a development process to a technology that is not fit for purpose. Immature neural network technology renders the auto’s environment only partially observable. Further, establishing truth in sensing is stochastic rather than deterministic. There are no deterministic switch contacts, temperatures, pressures, flows only a probability you are heading for a telephone pole or an on-coming UPS van (see video). Picture yourself in a train control system design review redolent with hard core players expecting failure stats in the 10^-6 to 10^-9 range and you turn up with an, “Aw shucks we’re 80% sure our packed 20 carriage passenger train is NOT about to travel through a stop signal. Let’s deploy the software in every day use and see if it works.” You’d be laughed out of the room. <span class="Apple-converted-space"> </span>Yet the new age auto industry players seem to think this is ok. Tesla offers a not fully validated version of self-driving as an expensive standard option. Have we got another “aw shucks” here? “Hey it sorta, kinda works but keep your hands on the wheel just in case.” Auto industry engineering standards seem to have backslid by orders of magnitude through rank ignorance. The whole concept of system validation has been ditched! The regulators seem to be asleep at the wheel.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">NEURAL NET IS A BAD METAPHOR IN THE SAFETY CRITICAL CONTEXT. Attempting to model the human brain’s capability for situational awareness with a neural net is a bridge too far. It reminds me of the early stages of manned flight where we had to get rid of the notion that aircraft wings had to flap. The breakthrough came with the realisation that the aerofoil was the core area for study. We need to find the analog of the aerofoil in brain simulation. That is, find metaphors that inform embedded pattern recognition AI designs that are deterministic rather than <span class="Apple-converted-space"> </span>stochastic. This will take a while. Some say it’s in the same bucket as nuclear fusion; thirty years away and always will be.</span></li></ol><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">SOLUTIONS</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">I know I’m barking at the moon here. AIs will continue to be deployed in control systems regardless of my objections UNLESS regulators wake up to the seriousness of this situation and:</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">DO THEIR JOB AND REGULATE!</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">PROHIBIT THE DEPLOYMENT OF ANY AI THAT CANNOT BE VALIDATED IN A SAFETY CRITICAL CONTROL SYSTEM</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Right now this means all of them.</span></p><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">PS:</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">If you’ve detect a level of passion in the above you’d be RIGHT ON! Call me a relic of 20th century life critical engineering, but I’ve seen this all before. In an accident of history when I graduated from university I fell in with a bunch of American chemical engineers who had just commenced to control chemical reaction kinetics directly with software. PDP-8s, assembler language, no code reviews, no hazard analysis, control programs tested on the plant (no off-line testing). All this in the the midst of the most stringent and effective safety culture I have ever experienced in a 46 year engineering career. How did we get away with it?</span></p><ol class="ol1" style="-webkit-text-size-adjust: auto;"><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">BISC + senior management were computer illiterate, they had no concept of the risks we were taking. Frankly neither did we. Functional safety engineering did not exist circa 1970. Personally I was having a fun time applying the advanced control algorithms that computers enabled. Our work was considered a black art. People pointed me out at company functions. “Oh so that’s the wizard.” Above all IT WAS SO COOL!</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">We got results. Product quality was up. Plant down time was down.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">There were no safety incidents traceable to computer control. A combination of good management and good luck. Some near misses were swept under the carpet.</span></li></ol><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">THEN … one of our number went too far! Having lined out the plant-embedded direct digital control system we started networking a supervisory computer for production reporting. You could also implement a more user-friendly plant operator interface with these machines.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">THEN … we attached modems to these computers so the plant engineer could monitor plant performance in real time from his living room.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">BUT … in the best BISC tradition it was also possible to change controller set points from your living room.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">SO pleased with himself was the actor who routinely did this, he presented his brilliance to the company’s crusty old corporate engineering manager</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">AND was told to “STOP THAT NOW BOY!”</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">It turns out that safe operation of a chemical plant requires on-site observibility of plant status.</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">REGULATION HAD FINALLY ARRIVED to curb the excesses of me and my BISCy mates.</span></p><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Abstracting this experience and projecting it onto the auto industry of 2022:</span></p><ol class="ol1" style="-webkit-text-size-adjust: auto;"><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Auto industry regulators seem to have no visibility of the risks companies like Tesla are taking with human life. Risks that would be unacceptable in most other industry sectors.</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Self-driving system developers are as ignorant of safe software engineering process as was my cohort circa 1970. Our excuse was: “aw shucks it was not codified then.” WELL IT CERTAINLY IS NOW!</span></li><li class="li1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal;"><span class="s1" style="font-size: 21.05px;">Recent self driving fatalities indicate the problem is worse than it was back then. In those days just proximity to a death at work would end your career. Fast forward to the recent past and you witness Elon getting away with responses to Tesla <span class="Apple-converted-space"> </span>crashes such as, “There are 1.2 million automotive deaths per year …” essentially, “s**t happens. Suck it up.” The narrative people like Elon are attempting to promulgate is “banning my software crushes innovation … some deaths are unavoidable (read acceptable).” <span class="Apple-converted-space"> </span>The professional engineer’s response (in case anyone has forgotten) is: THE ENGINEER’S DUTY IS TO APPLY SCIENCE FOR THE BENEFIT OF MANKIND. THE ONLY ACCEPTABLE DEATH RATE ATTRIBUTABLE TO A HUMAN ENGINEERED SYSTEM IS ZERO. Tesla owners are not in your army Elon. They did not sign up to die for Tesla motors.</span></li></ol><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">One characteristic of my 1970s control systems which does not project onto the present is visibility. Back in the day it was excellent. The promise of more precise control triggered a step change in spending on highly deterministic instrumentation and final control elements. Thirty percent of total plant capital cost was a good rule of thumb. </span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">In contrast, the visibility in so-called state of the art auto pilots is so bad they might as well be blind. Yet regulators continue to allow the unsuspecting public to drive these vehicles with software capable of killing them in the doesn’t-quite-work-yet state common in beta test. WHAT!!</span></p><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">SO I ask you, as with Diogenes the Cynic, who went around the sunlit streets of Athens, lantern in hand, looking for an honest man; where can I find a regulator with the integrity, passion and courage to grab Musk and his ilk by the lapels and direct them to</span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">STOP THAT NOW BOY!</span></p><p class="p2" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; min-height: 25.1px; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;"></span><br></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Thank you </span></p><p class="p1" style="margin: 0px; font-stretch: normal; font-size: 21.1px; line-height: normal; -webkit-text-size-adjust: auto;"><span class="s1" style="font-size: 21.05px;">Les</span></p><br><div dir="ltr">Managing Director<div>Chambers & Assoc. Pty Ltd. </div></div></body></html>