[SystemSafety] "FAA chief '100% confident' of 737 MAX safety as flights to resume"

[Pekka Pihlajasaari]

The intelligence community trades in imprecision and has recognised since at least the 1960s that prose with estimates of probability without a common understanding of the degree intended will fail in presenting its case. 

While this is not precisely a mapping from numbers to words, it does provide an function from words to numbers and its inverse may serve as such.

The appended link presents a scale proposed by a CIA analyst [1, 2, more readable Wikipedia entry at 3] to quantify estimates of probability. At no point does '100% confident' mean anything other than certainty. Instead, the intent appears to be one where prose descriptions can be reliably mapped to intervals interpreted along a scale of impossible (0%) through certain (100%). 

It remains unlikely that these intervals could be used when accumulated (as in tolerance stacking) or scaled (as in yield calculations) with any degree of reliability. Although tolerances are indicated for each element in the scale, the uncertainty of the estimate would almost certainly dominate the overall result of any compounded result.

The previous paragraph is an exemplar of how adjectives can influence interpretation and demonstrates how media demands for certainty are bound to fail. The publicity impact of claiming '100% confidence' appears to have been more important than the limit it implied. 

Richard Feynman's observation [4] in the conclusion of his dissenting view of the Challenger accident investigation board "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." remains immune to pressures from government, regulators and commercial interests.

Regards
Pekka Pihlajasaari
--
pekka at data.co.za	Data Abstraction (Pty) Ltd	+27 11 484 9664
--
1.	Words of estimative probability	https://www.cia.gov/library/readingroom/docs/CIA-RDP84B00506R000100070007-1.pdf 
2.	Words of estimative probability	https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/sherman-kent-and-the-board-of-national-estimates-collected-essays/6words.html
3.	Words of estimative probability	https://en.wikipedia.org/wiki/Words_of_estimative_probability
4.	Report of the PRESIDENTIAL COMMISSION on the Space Shuttle Challenger Accident Volume 2: Appendix F - Personal Observations on Reliability of Shuttle https://history.nasa.gov/rogersrep/v2appf.htm

-----Original Message-----
From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de> On Behalf Of Derek M Jones
Sent: 20 November 2020 17:58
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] "FAA chief '100% confident' of 737 MAX safety as flights to resume"

Bev,

> I don’t know about US speakers of our (apparently) common language. But I think English English speakers - and readers - would regard “100% confident” to mean “certain”. It does not seem to me to be a “quantification of confidence”, as you put it, but a declaration of absence of doubt.

I am not aware of any research that investigates assigning probabilistic or quantification words to probabilities.  However, there have been studies investigating assigning probabilities to words and phrases.

See Figure 6.7 and 6.8, also figure 2.57 might be of interest.
pdf+data here: http://knosof.co.uk/ESEUR/

Does anybody know of studies investigating numbers to words?

-- 
Derek M. Jones           Evidence-based software engineering
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety