[SystemSafety] Automobile Safety-Critical Kit (Bookout v. Toyota Motor transcript)

[Heath Raftery]

I read the transcript as best I could yesterday. To read it word for 
word would likely take a couple of days, so I skimmed. Interestingly, 
the excerpts in the EE Times article I did read verbatim - I think that 
section was not only of greatest interest to engineers, but probably 
ended up having the greatest impact on the court.

Mr Barr seems, based on the transcript, to be an excellent code 
reviewer. He's also an excellent communicator. He put together a very 
convincing case that the code was unsatisfactory. Not only was the 
quality of the code poor, their review and tracking procedures were 
sorely lacking. This is beneath what we would want to believe a car 
manufacturer adheres to.

On the other hand, I remain unconvinced the software had anything to do 
with the crash. The driver was 76 years old at the time. This crash was 
subject to an NTSB investigation, and investigators found no evidence 
that it was a software fault or a hardware fault. The crash recorder 
says the driver pushed the accelerator and was not pushing the brakes, 
and then the car was hit. Even Mr Barr's demonstration of a potential 
fault scenario (inject a bit flip, task dies, cruise control is resumed 
while the brake is lightly applied - acceleration continues past 
setpoint), while clearly a bug that should be fixed, is very hard to 
imagine as relevant to the crash.

I believe Mr Barr rightly convinced the court that the software 
development project lacked quality, and then, by virtue of some terrible 
cross-examination, he leads the jury to assume that the software bugs 
could have caused the accident.

I suspect, with no proof, that the code reached that state iteratively - 
I could well imagine it started with the best intentions and best 
practices. Then, as budgets dwindled, management got jumpy, and feature 
request/bug fixes got thrown in at the last minute, the code got messy 
and the review process fell away. The team probably consoled themselves 
by enormous system tests - in 1000's of km of testing, not one issue, so 
the crappy code works. It looks like rubbish, but it works.

Then, an old lady hits the accelerator instead of the brake, the code 
gets presented in court, a jury of laypeople looking for someone to 
blame are shocked at what actually runs the cars we drive and boom - 
Toyota is at fault in an accident causing death.

In the end I think people will have to start to expect software to cost 
more - we need these laborious (and rather dull, frankly) safety 
practices for two reasons: the first is that they can help build in 
quality (cue endless argument about whether they are the best way or 
even an effective way) and second because as engineers we're responsible 
for people's lives and sooner or later we need *evidence* that we've 
taken that into consideration.

It's an incredibly timely episode for our workplace, where we develop 
hardware and firmware for safety critical applications in another field 
entirely (mining) - there's this uneducated push to "go SIL" but no one 
appreciates the learning curve and the cost, and when push comes to 
shove, we just resort back to getting the job done as quickly as 
possible. That's starting to change.

Regards,
Heath

On 1/11/2013 5:30 AM, Chris Hills wrote:
> The complete transcript.
>
> http://veriloud.com/Barr_REDACTED.pdf
>
> *From:*systemsafety-bounces at lists.techfak.uni-bielefeld.de
> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] *On Behalf
> Of *Chuck_Petras at selinc.com
> *Sent:* 31 October 2013 17:57
> *To:* systemsafety at techfak.uni-bielefeld.de
> *Subject:* Re: [SystemSafety] Automobile Safety-Critical Kit (Bookout v.
> Toyota Motor transcript)
>
> Some excerpts from the court transcript...
>
> Toyota Case: Inside Camry’s Electronic Control Module,
> <http://www.eetimes.com/document.asp?doc_id=1319952>
>
>
> Chuck Petras, PE**