When Lightning Strikes Google Data Center

Aug 25, 2015

When Lightning Strikes Google Data Center

BY Chris Brunau

BCDR

A Google data center in Belgium fell victim to Mother Nature when a nearby power grid was struck by lightning four times.

According to the BBC, a number of disks damaged during the strike were eventually accessible, but some people permanently lost access to files.

“In a very small fraction of cases (less than 0.000001% of PD space in europe-west1-b), there was permanent data loss,” said an online statement from Google.

Google has apologized to all those affected, and says they have conducted a thorough analysis of the issue to identify contributing factors.

According to Google, on Thursday August 13, “four successive lightning strikes on the local utilities grid that powers our European datacenter caused a brief loss of power to storage systems which host disk capacity for GCE instances in the europe-west1-b zone. Although automatic auxiliary systems restored power quickly, and the storage systems are designed with battery backup, some recently written data was located on storage systems which were more susceptible to power failure from extended or repeated battery drain. In almost all cases the data was successfully committed to stable storage, although manual intervention was required in order to restore the systems to their normal serving state. However, in a very few cases, recent writes were unrecoverable, leading to permanent data loss on the Persistent Disk.”

With cloud to cloud backup through Backupify, you would also be able to protect your cloud data from accidental deletion or failure. Backupify goes above and beyond to make sure your cloud data is secure and protected.

A similar case happened with Datto partner, Neuse Networks in the United States. Even with top-of-the-line surge suppression, two lightning strikes took out the servers at Custom Assemblies. Luckily, Custom had Datto’s hybrid VM failover through the SIRIS solution. The Datto worked in both cases, even when the server went down.

Although the two situations were on different scales, Datto was able to save any information in Custom’s case.

“Our Datto system was a lifesaver,” says Kathy Jackl, Operations Manager at Custom Assemblies. “If Jim Ray and Neuse Networks had not had the foresight to insist that we implement a true disaster recovery solution—rather than mere data backup—the adverse financial impact of those two lightning strikes could have been significant.”

Subscribe to the Blog