[SystemSafety] State of the art for "safe Linux"

[Paul Sherwood]

On 2024-08-07 17:33, andrew at andrewbanks.com wrote:
> On Wednesday, August 7, 2024 10:38 AM, Paul Sherwood wrote:
> 
>>> Because we can **test**, without creating **tests**.
>>> We may have executed the code, but not created tests for it.
> 
> Indeed... mega-hours of nominal operation is fine, but serves no 
> purpose if
> 1. the test object is not appropriately specified, or inappropriately
> configured
> 2. the test environment is not specified, configured, nor repeatable
> 3. the test scenario is not specified, configured, nor repeatable
> 4. the expected result is not specified to an appropriate accuracy,
> precision, specificity and sensitivity
> 5. the achieved result is not favourably compared to the expected 
> result
> Etc
> 
> There is a lot more to testing than simply randomly executing it.

I agree with your points 1-5 as good practice (best practice? who really 
knows what's best?).

But to say that operation without those practices "serves no purpose" 
doesn't align with my experience. I've seen way too much supposedly 
"tested" software behave unexpectedly in operation.