[SystemSafety] Comparing reliability predictions with reality

[Robert P Schaefer]

hi, all,

being an old guy, I see there’s a new generation of software engineers out there who are trying to improve on previous generation’s efforts

(psychology and empirics) here are two references from which I’ve found hope:

https://osf.io/preprints/psyarxiv/tfjyw_v1
Hicks, C. M., & Hevesi, A. (2024, November 21). A Cumulative Culture Theory for Developer Problem-Solving.

and

Analyze This! 145 Questions for Data Scientists in Software Engineering
Andrew Begel, Thomas Zimmermann

ICSE 2014: Proceedings of the 36th International Conference on Software Engineering
Pages 12 - 23
https://doi.org/10.1145/2568225.2568233

still standing,
  bob s

> On Feb 24, 2025, at 8:10 AM, Derek M Jones <derek at knosof.co.uk> wrote:
> 
> All,
> 
> Having spent some time reading lots of papers on
> models of software reliability, based on
> cpu time between fault experiences, I have not found
> one that measures the accuracy of the models.  By
> accuracy I mean comparison of the prediction of time
> to next fault against actual time to next fault.
> 
> There is something of a cottage industry of papers
> that compare probability distributions fitted to historical
> data.  Researchers seem shy about taking the next step of
> comparing predictions of future faults.  Perhaps because
> the results are not very good?
> 
> Does anybody know of papers that compare predictions against
> actual?
> 
> Yes, lack of data is a perennial problem.
> 
> Some related discussion here
> https://shape-of-code.com/2025/02/23/deep-dive-looking-for-good-enough-reliability-models/
> 
> -- 
> Derek M. Jones           Evidence-based software engineering
> blog:https://shape-of-code.com
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety