<div dir="ltr"><div>Thanks Peter,</div><div><br></div><div>I found this is also available in correctly formatted <span class="" style="" id=":2kw.50" tabindex="-1">PDF</span> at <a href="https://arxiv.org/abs/1911.11034"><span class="" style="" id=":2kw.51" tabindex="-1">https</span>://<span class="" style="" id=":2kw.52" tabindex="-1">arxiv</span>.org/abs/1911.11034</a></div><br><div>Victoria <span class="" style="" id=":2kw.53" tabindex="-1">Krakovna</span> also maintains an AI safety resources page which includes database records of AI failures. This is at <a href="https://vkrakovna.wordpress.com/ai-safety-resources/"><span class="" style="" id=":2kw.54" tabindex="-1">https</span>://<span class="" style="" id=":2kw.55" tabindex="-1">vkrakovna</span>.<span class="" style="" id=":2kw.56" tabindex="-1">wordpress</span>.com/<span class="" style="" id=":2kw.57" tabindex="-1">ai</span>-safety-resources/</a></div><div><br></div><div>Artificial Intelligence, Machine Learning and Deep Learning are evolving fields, especially when it comes to dependability aspects. The <span class="" style="" id=":2kw.58" tabindex="-1">Cyber</span>-Physical Systems and Cognitive Systems domain are sort of at the stage when IT was rapidly introduced to OT systems back around the turn of the century (sounds so old to say that).</div><div><br></div><div>I think it is a bit too early to dismiss ML failures as just software or systematic failures. True ML failures like systematic failures
(<span class="" style="" id=":2kw.59" tabindex="-1">IEC</span> 61508-4 3.6.6)
are "related in a deterministic way to a certain cause "; in this case the "learning" process; environment; and data. Can it be "eliminated by a<br>modification of the design or of the manufacturing process"? It also depends, somewhat, on whether learning is part of the design process and then when validated, locked <span class="" style="" id=":2kw.60" tabindex="-1">baselined</span> or whether the system continues to learn and thus acquires more "systematic faults".</div><div><br></div><div>AI is an interesting field an not necessarily well thought through on the possible side effects, as exhibited in other fields of application. In my opinion,
with disruptive technology,
governments act
a bit like the Emperor in Andersen's fairy tale; they are wowed by the buzz but not willing to contemplate the risks. The Australian Government learnt this with its crazy <span class="" style="" id=":2kw.61" tabindex="-1">Robodebt</span> Scheme for welfare fraud which, after many years and much damage to vulnerable people, they had to admit its failure. The Australian Human Rights Commission (yes we do have one) released a report on this today at <a href="https://tech.humanrights.gov.au/sites/default/files/inline-files/TechRights2019_DiscussionPaper_Summary.pdf"><span class="" style="" id=":2kw.62" tabindex="-1">https</span>://tech.<span class="" style="" id=":2kw.63" tabindex="-1">humanrights</span>.gov.<span class="" style="" id=":2kw.64" tabindex="-1">au</span>/sites/default/files/<span class="" style="" id=":2kw.65" tabindex="-1">inline</span>-files/TechRights2019_<span class="" style="" id=":2kw.66" tabindex="-1">DiscussionPaper</span>_Summary.<span class="" style="" id=":2kw.67" tabindex="-1">pdf</span></a></div><div><br></div><div>For those that want more on AI and ML failures on humans and ethics, there is a good read "Made by humans" by Ellen Broad.</div><div><br></div><div>I don't think we need to call in Sarah Connor just yet, though ;-)<br></div><div><br></div><div>Bruce Hunter<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 16 Dec 2019 at 00:35, Peter Bernard Ladkin <<a href="mailto:ladkin@causalis.com">ladkin@causalis.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">For those interested in how current analytical methods can adapt to new SW techniques, here is a<br>
taxonomy of ML failure modes:<br>
<br>
<a href="https://docs.microsoft.com/en-us/security/failure-modes-in-machine-learning" rel="noreferrer" target="_blank">https://docs.microsoft.com/en-us/security/failure-modes-in-machine-learning</a><br>
<br>
It was noted by Bruce Schneier, who is at Berkman-Klein, where some of the authors are, and<br>
published in his Crypto-Gram today, which is where I got it from.<br>
<br>
PBL<br>
<br>
Prof. Peter Bernard Ladkin, Bielefeld, Germany<br>
MoreInCommon<br>
Je suis Charlie<br>
Tel+msg +49 (0)521 880 7319 <a href="http://www.rvs-bi.de" rel="noreferrer" target="_blank">www.rvs-bi.de</a><br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
The System Safety Mailing List<br>
<a href="mailto:systemsafety@TechFak.Uni-Bielefeld.DE" target="_blank">systemsafety@TechFak.Uni-Bielefeld.DE</a><br>
Manage your subscription: <a href="https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety" rel="noreferrer" target="_blank">https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety</a></blockquote></div>