« RandomX Java Package Updated | Main | Reading List: Drug Lord »

Monday, August 28, 2017

Flash, Bang!

Last Thursday, 2017-08-24, was interesting. I was programming away when, as often occurs in the mid-afternoon here in the summer, thunderheads boiled up above the Jura, the sky darkened, and before long a full-on thunderboomer complete with high winds, torrential rain, and plenty of flashes and bangs was underway.

Then, flash, bang!  When you perceive them at exactly the same time, it's never a good sign. Instantaneously, the auxiliary monitor on my development machine emitted a crack and went black. The UPS units all went on battery, but quickly came back on line. I suspect the event they detected was not a power outage but a transient due to the lightning strike. After about five seconds, the monitor lit back up as if nothing had happened.

So far, so good. Further along, not so good. The phone next to the computer, which is connected to Fourmilab's Alcatel OmniPCX phone central was completely dead: even the LCD display was blank. All of the other phones connected to the central were equally hors de combat. The main Fourmilab fibre optic connection to the Internet wasn't perturbed at all, but the backup Swisscom ADSL connection was dead, and its router responded by butting in to new connections and diverting them to its "landing page" until I pulled the plug.

This isn't the first time something like this has happened. When you live on a plateau 806 metres above sea level just downwind of the first serious mountain range moist air encounters after crossing France and being heated, summer thunderstorms are part of the deal. (Although I don't keep detailed records, I think this is about the eighth time Fourmilab has been struck by lightning resulting in damage to electronics. Once is chance; twice is coincidence; three times is enemy action; —eight— you must've really made old thunderhammer quite irate.)

All of the power and telephone lines are buried, and protected with state of the art lightning arrestors, and all non-resistive electrical loads are connected to UPS units. The problem is nearby strikes which are conducted to ground by the protection, but which, in doing so, induce currents in data cables parallel to them. You can do everything right, but when you're talking about hundreds of thousands of amperes, all of your remediation does about as much good as a tinfoil hat.

Lightning rods on buildings don't help. They may protect the building, but the cone of protection doesn't extend sufficiently far to guard against ground strikes which get into telephone wires. You could put up lightning protection masts like they use around rocket launch pads, but it would be difficult to get the neighbours to approve.

Here is what happens when lightning gets into a data cable between two buildings. First. here's the connector into which the cable was plugged.


It was port 2 that was hit. Note how the metal at the top of the connector has been melted and the conductors blackened by smoke. The smoke above the connector was the tip-off, and more visible to the eye than in this picture.

Here's the cable:


You can see the smoke staining the plastic of the connector. There are also smoke stains on the metal sides of the connector, but they don't show up well since it's a specular reflector.

I'd like to say that everything has been resolved and all is well, but it's not. There's a lot more time to be consumed in remediation of this event. What can you do to keep this from happening to you? Don't get struck by lightning!  Other than moving to somewhere the risk is lower, there's little more that can be done.

Posted at August 28, 2017 23:50