Eight major IT disasters in 2024

Like most years, 2024 has seen a series of IT disasters, some of which were forgotten in a few days or weeks, but others that had lasting effects and, in one case, cost billions of dollars.

For this list, we’ve mostly ignored data breaches, and readers interested in such incidents can refer to other lists. We’ve already published a list of recent AI failures, several of which happened in the past year. For example, fast food giant McDonald’s abandoned an AI-based ordering system in June because it kept mistakenly adding food to customers’ bills.

The mother of IT disasters

In mid-July, a software update glitch from cybersecurity vendor CrowdStrike caused about 8.5 million computers running Windows to crash to a blue screen of death and then get stuck in a repetitive boot loop, rendering Windows computers that were stuck in an endless boot loop virtually useless except for use as doorstops or paperweights.

Hospitals, flight reservation centers, emergency response centers and public transportation services were all affected by the outage, which caused hundreds of flight cancellations and other problems 24 hours after it was first reported. The outage was estimated to have cost more than $5 billion.

CrowdStrike blamed the outage on a vulnerability in its software testing tools that appeared in a sensor configuration update released to Windows systems on July 19. The vulnerability was in a vulnerability signature update called Rapid Response Content, which is less rigorously tested than some of CrowdStrike's other updates.

The fallout from the outage was swift and continues to this day, prompting some CIOs to reconsider their reliance on cloud infrastructure and prompting Microsoft to pay more attention to kernel-level access permissions for other software packages.

Meanwhile, Delta Airlines filed a $500 million lawsuit against CrowdStrike and Microsoft and is reconsidering its use of Microsoft products.

Millions of missed calls

While the scale of the CrowdStrike outage was huge, it was overshadowed in terms of the number of people affected by a February outage at AT&T Mobility that affected 125 million mobile devices in the U.S. The outage lasted more than 12 hours and prevented about 92 million calls from being completed, including 25,000 emergency 911 calls, the Federal Communications Commission said. The massive outage was caused by a misconfiguration of the equipment.

It took AT&T nearly two hours to roll back the network changes, but restoration of full service took at least 12 hours because the mobile operator's device registration system was overwhelmed by re-registration requests, the FCC said.

Then in June, AT&T customers reported another service outage. Reports of service outages began to surge at 1 p.m. (ET) on June 4, then dropped off around 6 p.m. Areas around New York City, Chicago, Philadelphia, Dallas, Pittsburgh, and Indianapolis were apparently affected.

McDonald's IT Problems

Besides the AI ​​ordering system that thought a customer wanted more than 200 Chicken McNuggets, McDonald's has had more IT problems. In March, a massive outage that affected credit card orders (both online and at self-service terminals) lasted about 12 hours.

McDonald's restaurants in the Far East, Europe, the United States and Australia have reported problems with credit card payments, with the problem blamed on a daunting third-party configuration change. The company's global chief information officer did not provide details but noted that the outage was not related to a cybersecurity attack.

The Dangers of Third-Party Software Updates

McDonald's is not the only company to suffer outages in its point-of-sale (POS) systems. In the UK, supermarkets Tesco and Sainsbury's, as well as bakery chain Greggs, have experienced problems with POS systems operated by third parties at the same time McDonald's reported similar issues. In most cases, the problems were resolved within a working day, but the companies were unable to process credit card payments during this time.

In some cases, affected companies reported that the problems were related to software updates, raising questions about the reliability of third-party POS providers.

Chatbots out of control

You'd think people would learn something after the same thing keeps happening. In February, Microsoft launched an investigation into its Copilot AI chatbot after reports surfaced on social media that it taunted users who were considering suicide. Microsoft found that the bad responses were the result of prompt injection attacks, in which users were able to override safety controls in large language model AIs. Microsoft said the bad responses were limited to a handful of replies.

This isn't the first chatbot Microsoft has run amok. In early 2023, an AI chatbot bundled with Bing began showing affection to some users and insulting others, calling them ugly and comparing them to Hitler.

Back in 2016, Tay, an experimental AI chatbot released by Microsoft on Twitter, expressed support for genocide and Nazism. We sense a trend here.

The February investigation into Copilot isn't the only problem Microsoft's AI assistant has run into. In November, Microsoft rolled out new tools to prevent Copilot from oversharing data, such as confidential employee information.

Scholarship farce

In March, the U.S. Department of Education said it had discovered errors in the calculation of financial aid for hundreds of thousands of college students, resulting in delays in the payment of financial aid.

The Education Department blamed the problem on a vendor working for the federal government that incorrectly calculated the bursary formula, affecting more than 200,000 students.

The calculation error occurred as the Education Department overhauled the Free Application for Federal Student Aid, or FAFSA, which is used to determine student eligibility for federal Pell Grants and other financial aid.

The provider apparently failed to factor some assets, such as investments, into the financial need of some students, causing scores to show their financial need was higher than it actually was.

Meanwhile, the Department of Education’s overhaul of the FAFSA form has also caused delays in the financial aid application process. While the form is typically filled out in October, it wasn’t ready until late December, and the Department of Education didn’t begin processing and sending it to states and colleges until March. The form was only sporadically available in late December and early January.

The Department of Education encountered several glitches and bugs while processing the form, including one that prevented parents without Social Security numbers from filling out the form.

It's unclear how many of these problems were pure IT disasters or a mix of IT disasters and user errors, which appear to be a bit of both.

Friendly fire

In February, Chinese PC maker Acemagic admitted to selling machines with malware installed on them.

The admission came after YouTuber The Net Guy discovered malware while testing Acemagic mini PCs, which came pre-installed with the Backdoor.Bladabindi malware as well as RedLine Stealer.

In a bizarre explanation, the company blamed the problem on developers who tried to improve boot time with some software modifications.

The real terminator

As you might expect, the UK’s Post Office, a government-run delivery service, fired more than 700 employees in late 2023 and early 2024 on the advice of its Fujitsu-built Horizon IT system, which, it turned out, had falsely accused these former employees of stealing money from the service, falsely claiming that funds were missing from accounts they controlled.

Some news reports said the Horizon system, which was installed as early as 1999, did not share documentation of known errors with its postal regulators. In addition, postal employees have complained for years about erroneous reports of missing funds.

The Post Office attempted to move away from Horizon and onto the cloud in 2023, but the effort failed and cost it £31 million. In late January, Fujitsu was suspended from bidding for UK government contracts.