19 of the worst IT outages in 2019 – A Recap of Being Let Down

You know how at the end of every year lists are created intending to reflect, usually, 10 of the year’s best examples of a particular phenomenon or event? Well, we’ve also compiled a list but ours has a couple of twists.  One, while most lists are described as the “best of,” ours can be described as the “worst of.” Two, our list doesn’t have 10 items, but 19, in line with the number of years into the 21^st century.  So, without too much further ado, we present to you our list of 19 of the worst IT outages of 2019. But first…

What makes an outage bad enough to warrant a place on our list? We decided on three criteria to use for listing an outage: 1. length of the outage; 2. number of people/systems affected; and, 3. cost of the outage. We’ll admit, though, that a couple made it onto the list because they were unusual, sort of groundbreakers.

Outages seem to escape any clear classification; they span industry segments (we’ve seen dramatic outages in financial services, social media, airlines, telco, retail, etc.), they cover both new and well established infrastructures (from pure public-cloud infrastructure, to “traditional” IT, and any hybrid permutation in between), and impact enterprises of all sizes. In other words, no one, not even cloud providers themselves, is immune against outages!

To help you place the outages in the right context of reasons why whole environments went down, we’ll review the causes leading to the outages covered here. Generally, the outages fall into three categories: technical software or hardware issues; unusually high traffic / load; cyberattacks and/or ransomware.

IT outages due to technical software or hardware issues  

As our table below shows, software or hardware issues are the key reasons organizations experience outages/unavailability and performance disruptions.  Not every enterprise will let the public know why its site went down and will often issue a non-informative reason, attributing the outage to a software or computer “glitch.” Usually, “glitch” can be interpreted as a misconfiguration or a failure of redundant systems to failover (see our post on this).  As a rule, when a system becomes unavailable, a secondary, redundant, site should take over operations. Many outages occur when the transition to the secondary site fails to take place.

 IT outages due to unusually high demand / traffic / load

When sites experience unusually high traffic and do not have enough bandwidth and/or extra load fails to be diverted to the additional servers enterprises undoubtedly have available, an outage occurs. These kinds of outages occur at all types of sites and in different industries. Outages can grab headlines when they result after a great buildup that creates expectations among users – for example, on Black Friday. But, man/woman does not live by shopping alone, and not being able to access critical and potentially life-saving information also angers people.

Both online stores and online customers look forward to special sale days. The stores rake in many millions of dollars in sales while shoppers enjoy deep discounts on desired items. Online sites are deluged customers requesting access to their sites but some apparently don’t have the bandwidth in place to accommodate the amount of traffic they receive, causing their site to crash. In 2018 we reported on the major retailers that saw their sites go down and miss out on hundreds of millions of dollars in sales. These included Walmart, Lowe’s, Lululemon, and J. Crew, and many others. It’s been estimated that for every hour of the site is down, a retailer will lose 4% of the day’s sales.

This year, H&M experienced outages on Thanksgiving eve and Thanksgiving Day while Nordstrom Rack shoppers reported technical issues and Home Depot had frustratingly slow load times. We can expect more reports. In the meantime, the most upsetting outage was at Costco.

IT outages due to cyberattack and/or ransomware

A recent survey found that more senior managers at enterprises were worried about a cyberattack (53%) than an IT outage (36%) or network failure (24%). Twenty percent of managers in the UK revealed they had already experienced 3-4 cyberattacks in the preceding 12 months and suffered the usual fallout from such attacks.

Ransomware has caused significant problems for major corporations and cut off access to core data. The financial industry recently simulated a fictional scenario highlighting what would happen if a ransomware attack targeted the biggest financial institutions, taking critical parts of the global financial system offline.

Indeed, enterprises and organizations of different types from shipping corporations and telecoms to municipalities and healthcare providers and hospitals have been suffering from cyberattacks, frequently in the form of ransomware attacks. Malicious actors attack targets in order to sow disruption and extort large payments from victims so they can access their key data. To guard against attacks organizations must build cyber resilient environments and have cyber recovery solutions in place. Solutions for cyber recovery differ from those for disaster recovery, which are appropriate for catastrophes resulting from events such as earthquakes, floods, or human error.

The proliferation of attacks has already led security experts and government regulators to declare that it is no longer sufficient to implement comprehensive security solutions alone. Enterprise must now ensure that their IT environment is resilient, which means that under circumstances of a cyber event/incident or attack, the enterprise must be able to continue delivering services.  Organizations must ensure their critical data is not compromised and is recoverable, and to do so, they must build cyber resilient environments protected by cyber-resilience and recovery solutions.

Summary

The IT outages reviewed here impacted hundreds of millions of people worldwide. Many other IT outages occurred for a variety of reasons. These were simply some standouts. It’s clear that to avoid such incidents, organizations should implement solutions which proactively and continuously keep IT and InfoSec up to date on the status of IT resilience and cyber resilience.

Our solutions help organizations around the world to prevent the vast majority of outages; 6 of the top 10 banks in the U.S. use our resilience solutions.

Contact us to learn more.