[SystemSafety] The Worsening Cosmic Ray Situation

[Todd Carpenter]

CP> ECC memory anyone?

Definitely a useful tool to help reduce, although not eliminate, the
issue. For example, it won't catch an SEE occurring within an ALU. Those
are expensive to prevent; one aerospace company reduced this particular
problem for a secure processor by voting the output of triplex ALUs.
That catches many issues, but not metastable events that lead to
Byzantine states. Point being, since we can only reduce certain issues,
not eliminate them, we therefore must construct systems that tolerate
some uncertainty.

This reminds me of the great discussion back in November about the
assumptions that software formal methods make about the underlying
machine model [1]. Back in November, SEEs, Rowhammer, and Intel's
management-engine backdoors (and ARM's CoreSight) were examples of
obvious flaws in those assumptions. Meltdown and Spectre were published
since then, but I suspect few of us on this list would admit to
tolerating super-scalar architectures anywhere near their control systems.

As yet another aside to this, many modern useful systems rely on
embedded flash cards (SD, SDHC, etc). In the olden days, memories were
pretty straightforward: you applied an address to one side, and the data
value stored at that memory popped out the other side.  Unfortunately,
things are no longer that conceptually straightforward. These modern
"memories" are now better thought of as "embedded systems with storage."
Even that tiny little micro-SD in your phone has a microcontroller on it
running all sorts of software which handles multiple protocols,
remapping bad blocks of memory, etc. Some companies are better than
others at writing that software, and no, it is not an industry standard
skill. Traditional EDAC (for example, 32-bit CRC) might not give you the
coverage you think it does, because locality of burst errors might be
spread in unanticipated ways on the physical medium, allowing errors to
accumulate beyond EDAC's ability. Many of the consumer-grade products do
not even remotely exhibit the property "Good EDAC." Also, some companies
are better than other at matching the microcontroller to the memory
process technologies. What we're seeing is that these microcontrollers
and software on these things are susceptible to SEEs and the devices
fail. Takeaway: buy good grade memories for both your personal and
company devices, and back up data you care about. For your safety
critical systems, evaluate these microcontrollers and their embedded
software; you can no longer assume that the devices are passive and
solely under the control of the software running on the processor.

-TC

<silly_diversion> [1] Contributors included David Mentré, David Crocker,
Derek Jones, and Daniel Kästner. The first initial seems to correlate to
interest in this topic? It's like the Chicken Pox Name Statistics.
https://xkcd.com/1950/ </silly_diversion>


On 3/7/2018 4:59 PM, Chuck_Petras at selinc.com wrote:
> ECC memory anyone?
>
> The Worsening Cosmic Ray Situation
> http://spaceweather.com/archive.php?view=1&day=05&month=03&year=2018[spaceweather.com]
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__spaceweather.com_archive.php-3Fview-3D1-26day-3D05-26month-3D03-26year-3D2018&d=DwMGaQ&c=zVFQZQ67ypsA9mYKSCqWmQHiVkCCaN-Gb60_N6TVnLk&r=zCwDz0h_ezUCVpbXoLT-zh0iTVdbymfdnT16kGAgelNE5W_nOFK-pESbjJCRy2gv&m=tOOBoe8GttFALe1zsz2QjjwNdHMq2zDiZoVruOpqT6A&s=mG18cd6baIpMrKYZUb1vISf3m8CZMraqP6h-XLavswY&e=>
>
>
> "The problem is, as the authors note in their new paper, the shield is
> weakening: 'Over the last decade, the solar wind has exhibited low
> densities and magnetic field strengths, representing anomalous states
> that have never been observed during the Space Age. As a result of
> this remarkably weak solar activity, we have also observed the highest
> fluxes of cosmic rays.'"
>
> "But even on Earth the increase is being felt. Spaceweather.com and
> the students of Earth to Sky Calculus have been launching space
> weather balloons to the stratosphere almost weekly since 2015. Sensors
> onboard those balloons show a 13% increase in radiation (X-rays and
> gamma-rays) penetrating our planet's atmosphere"
>
> "X-rays and gamma-rays detected by these balloons are 'secondary
> cosmic rays,' produced by the crash of primary cosmic rays into
> Earth's upper atmosphere. They trace radiation percolating down toward
> our planet's surface. The energy range of the sensors, 10 keV to 20
> MeV, is similar to that of medical X-ray machines and airport security
> scanners.
>
> "How does this affect us? Cosmic rays penetrate commercial airlines,
> dosing passengers and flight crews so much that pilots are classified
> by the International Commission on Radiological Protection as
> occupational radiation workers. Some research shows that cosmic rays
> can seed clouds and trigger lightning, potentially altering weather
> and climate. Furthermore, there are studies ( #1, #2, #3, #4) linking
> cosmic rays with cardiac arrhythmias in the general population."
>
>
> Chuck Petras, PE**
> Schweitzer Engineering Laboratories, Inc
> Pullman, WA  99163  USA
> http://www.selinc.com <http://www.selinc.com/>
>
> SEL Synchrophasors - A New View of the Power System
> <http://synchrophasor.selinc.com <http://synchrophasor.selinc.com/>>
>
> Making Electric Power Safer, More Reliable, and More Economical (R)
>
> ** Registered in Oregon.
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180308/9d4fcb34/attachment-0001.html>