<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div><br>
</div>
For what it’s worth, OWASP (Open Worldwide Application Security Project, www.owasp.org) released their “OWASP Top 10 for LLM”. You can find the document here:
<div><br>
</div>
<div><br>
</div>
<div>
<div style="display: block;">
<div style="-webkit-user-select: all; -webkit-user-drag: element; display: inline-block;" class="apple-rich-link" draggable="true" role="link" data-url="https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf">
<a style="border-radius:10px;font-family:-apple-system, Helvetica, Arial, sans-serif;display:block;-webkit-user-select:none;width:228px;user-select:none;-webkit-user-modify:read-only;user-modify:read-only;overflow:hidden;text-decoration:none;" class="lp-rich-link" rel="nofollow" href="https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf" dir="ltr" role="button" draggable="false" width="228">
<table style="table-layout:fixed;border-collapse:collapse;width:228px;background-color:#E5E6E9;font-family:-apple-system, Helvetica, Arial, sans-serif;" class="lp-rich-link-emailBaseTable" cellpadding="0" cellspacing="0" border="0" width="228">
<tbody>
<tr>
<td vertical-align="center" align="center"><img style="width:228px;filter:brightness(0.97);height:322px;" width="228" height="322" draggable="false" class="lp-rich-link-mediaImage" alt="preview.png" src="cid:4ABAA420-B74E-4EAA-9846-0878173BCB00"></td>
</tr>
<tr>
<td vertical-align="center">
<table bgcolor="#E5E6E9" cellpadding="0" cellspacing="0" width="228" style="font-family:-apple-system, Helvetica, Arial, sans-serif;table-layout:fixed;background-color:rgba(229, 230, 233, 1);" class="lp-rich-link-captionBar">
<tbody>
<tr>
<td style="padding:8px 0px 8px 0px;" class="lp-rich-link-captionBar-textStackItem">
<div style="max-width:100%;margin:0px 16px 0px 16px;overflow:hidden;" class="lp-rich-link-captionBar-textStack">
<div style="word-wrap:break-word;font-weight:500;font-size:12px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-topCaption-leading">
<a rel="nofollow" href="https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf" style="text-decoration: none" draggable="false"><font color="#272727" style="color: rgba(0, 0, 0, 0.847059);">OWASP-Top-10-for-LLMs-2023-v1_0</font></a></div>
<div style="word-wrap:break-word;font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-bottomCaption-leading">
<a rel="nofollow" href="https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v1_0.pdf" style="text-decoration: none" draggable="false"><font color="#808080" style="color: rgba(0, 0, 0, 0.498039);">PDF
Document · 33.6 MB</font></a></div>
</div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</a></div>
</div>
<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Quote (page 2):
<div><br>
</div>
<div>“<i>The frenzy of interest of Large Language Models (LLMs) following of mass-market pre- trained chatbots in late 2022 has been remarkable. Businesses, eager to harness the potential of LLMs, are rapidly integrating them into their operations and client
facing offerings. Yet, the breakneck speed at which LLMs are being adopted has outpaced the establishment of comprehensive security protocols, leaving many applications vulnerable to high-risk issues.</i></div>
<div><i><br>
</i></div>
<div><i>The absence of a unified resource addressing these security concerns in LLMs was evident. Developers, unfamiliar with the specific risks associated with LLMs, were left scattered resources and OWASP’s mission seemed a perfect fit to help drive safer
adoption of this technology.</i>”</div>
<div><br>
</div>
<div><br>
</div>
<div>Later on that same page:</div>
<div><br>
</div>
<div>“<i>Over the course of a month, we brainstormed and proposed potential vulnerabilities, with team members writing up 43 distinct threats. Through multiple rounds of voting, we refined these proposals down to a concise list of the ten most critical vulnerabilities.</i>”</div>
<div><br>
</div>
<div>I, personally, wish they hadn’t stopped at just the top 10. I would surely like to know what the other 33 vulnerabillities are all about. But at least it seems people are taking this seriously.</div>
<div><br>
</div>
<div>
<div><br>
<div>Cheers,</div>
<div><br>
</div>
<div>— steve</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div>
<div>
<div style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<br>
<div><br>
<div>On Aug 3, 2023, at 7:48 PM, Les Chambers <les@chambers.com.au> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div>Peter<br>
Your comment:<br>
"Martyn already observed on 2023-06-27 that there are legal requirements which <br>
constrain deployment of safety-related systems. That legal requirement in the <br>
UK and Australia is 77 years old. Your question seems to be suggesting that you <br>
somehow think it, and other constraints, might no longer apply. Well, they do. <br>
As Martyn said "AI doesn't change that.<br>
In the UK or Australia, developer and deployer must reduce risks ALARP."<br>
<br>
. is righteous . that is, if decreed by a king ruling by fiat (Latin for "let <br>
it be done").<br>
<br>
Legal requirements are one thing, usually coming into play to the right of <br>
"bang"; keeping the public safe in the first place is another more important <br>
issue.<br>
The interesting question is, how does one PROVE (to an auditor or a judge) that <br>
one has reduced risks ALARP if one's delivered system's behaviour is initiated <br>
from a neural network? A dataset that cannot be interpreted, verified or <br>
validated thoroughly in process, and that changes after delivery. AI <br>
aficionados admit they don't understand why NNs can work so well or fail so <br>
unpredictably. <br>
Witness: https://dawnproject.com/<br>
<br>
Case study: Elaine Herzberg, killed by a self-driving Uber in Tempe, Arizona in <br>
2018. The system did not classify her as a pedestrian because she was crossing <br>
without a crosswalk; the neural net did not include consideration for <br>
jaywalking pedestrians. <br>
<br>
These systems are famous for not knowing what they don't know and imposing <br>
their ignorance on the real world. Hannah Arendt was prescient: "It's not so <br>
much that our models are false, but that they might become true"<br>
<br>
Imagine yourself as an expert witness supporting Tesla in a similar situation. <br>
What section, subsection or footnote of IEC 61508 or ISO 26262 - or other <br>
standard - would you cite to prove Elon had applied best practice in his <br>
development life cycle?<br>
<br>
Or, if you cannot pony up, would you agree that these standards are no longer <br>
fit for purpose in regulating the development of AI-integrated Safety-Critical <br>
systems?<br>
<br>
And furthermore, please explain the purpose of these standards, if they cannot <br>
be instrumental in stopping the murder for money currently occurring on US <br>
roads? <br>
<br>
Les<br>
<br>
PS: I note that Tesla's full self-driving (FSD) feature is available in the UK <br>
as well as the US. It is not available in Australia or Germany. <br>
<br>
---------------------------<br>
<blockquote type="cite">On 2023-08-03 02:32 , Les Chambers wrote:<br>
<blockquote type="cite"><br>
Can anyone on this list refer me to where in the standards one can obtain<br>
guidance on how to engineer such a system safely?<br>
</blockquote>
<br>
That seems to be a question with a completely obvious answer.<br>
<br>
Martyn already observed on 2023-06-27 that there are legal requirements which <br>
</blockquote>
constrain deployment <br>
<blockquote type="cite">of safety-related systems. That legal requirement in the UK and Australia is
<br>
</blockquote>
77 years old. Your <br>
<blockquote type="cite">question seems to be suggesting that you somehow think it, and other
<br>
</blockquote>
constraints, might no longer <br>
<blockquote type="cite">apply. Well, they do. As Martyn said "AI doesn't change that."<br>
<br>
In the UK or Australia, developer and deployer must reduce risks ALARP.<br>
<br>
How do you go about engineering any system such that risks are reduced ALARP, <br>
</blockquote>
say in the UK? You <br>
<blockquote type="cite">follow sector-specific functional safety standards if there are some, as well
<br>
</blockquote>
as the engineering <br>
<blockquote type="cite">functional safety standard for E/E/PE systems, which is IEC 61508. This
<br>
</blockquote>
approach is regarded by the <br>
<blockquote type="cite">regulator, at least in the UK, as appropriate to fulfill the ALARP
<br>
</blockquote>
requirement (although of course <br>
<blockquote type="cite">the courts are the final arbiters of that).<br>
<br>
PBL<br>
<br>
Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany<br>
Tel+msg +49 (0)521 880 7319 www.rvs-bi.de<br>
</blockquote>
<br>
<br>
<br>
--<br>
Les Chambers<br>
les@chambers.com.au<br>
<br>
https://www.chambers.com.au<br>
https://www.systemsengineeringblog.com<br>
<br>
+61 (0)412 648 992<br>
<br>
<br>
_______________________________________________<br>
The System Safety Mailing List<br>
systemsafety@TechFak.Uni-Bielefeld.DE<br>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety<br>
</div>
</div>
</div>
<br>
</div>
_______________________________________________<br>
The System Safety Mailing List<br>
systemsafety@TechFak.Uni-Bielefeld.DE<br>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety</div>
</div>
<br>
</div>
</div>
</div>
</div>
</body>
</html>