As the world recovers from the largest IT outage in history, it shows the danger of one point of failure in IT infrastructure
A global IT failure wreaked havoc on Friday, grounding flights and disrupting everything from hospitals to government agencies. Over all the chaos hung a question: how did a flawed update to Microsoft Windows software bring large swaths of society to a screeching halt?
The problem originated with an Austin, Texas-based cybersecurity firm called CrowdStrike, relied upon by most of the global technology industry, including Microsoft, for its Falcon program, which blocks the execution of malware and cyber-attacks. Falcon protects devices by securing access to a wide range of internal systems and automatically updating its defenses – a level of integration that means if Falcon falters, the computer is close behind. After CrowdStrike updated Falcon on Thursday night, Microsoft systems and Windows PCs were hit with a “blue screen of death” and rendered unusable as they were trapped in a recovery boot loop.
Microsoft is a juggernaut with significant market power, dominating cloud-computing infrastructure across Europe and the United States. So it wasn’t just computers that were affected, but servers and a host of other systems as well. Overwhelming requests from users, devices, services and businesses ushered in a cascading series of failures with Microsoft products – namely Azure Cloud and Microsoft 365. Failures plaguing Azure led to additional but separate disruptions with 365 services. A giant clusterfuck ensued.
No… the Crowdstrike debacle primarily shows the dangers of today’s corporate culture in software development.
Ship as fast as possible, fix issues later if necessary…
Yup. Push to prod!
This is the Boeing debacle in software land. Kill the engineering and pay the executives. QA? Testing? Strict standards? People? Naaah, more conferences! More logos on F1 cars!
I totally agree. But without knowing a bit more about the specifics, I can’t help but think that just maybe… The updating mechanism could have perhaps just rolled back an update if it caused a bsod?
Seems like that infrastructure is really the biggest oversight and people would have been none the wiser.
Also surprised just how many things are running windows. I thought for sure the self checkout registers would have been some embedded Linux system.
I disagree. You are correct that the cause of the fuck up is because of bad development practices. However, if every firm is being reckless with development, but only one out of a myriad of competing firms fucks up because of it, maybe you’d take one airline or hospital network offline or something like that.
It’s only because of consolidation and market monopolization of the sector, that an outage at such a global scale was even possible to begin with.
You’re partly right as is the article.
Centralization is dangerous for security, innovation and cost (monopolies, duopolies).
Am I missing something? I thought the outage was caused by CrowdStrike and had nothing to do with Microsoft or Windows?
The article actually talks about Azure which was using CrowdStrike internally so their point is valid but the headline is absolutely wrong. Azure is nowhere near a monopoly and it ends up implying that Windows, now Azure was the issue they’re describing.
This is the typical Guardian sensationalism. Gotta make it look like it was Microsoft’s fault, although this one is square on CrowdStrike’s head. Imagine if a security update for a remote administration tool caused an on-boot kernel panic on every linux server in the world…
Microsoft also had issues yesterday
Microsoft’s use of CrowdStrike meant that a significant number of their cloud and SaaS offerings also failed, impacting users who likely didn’t know what CrowdStrike was.
Only systems running CloudStrike were affected, but all systems were Windows-based as that is the only OS it works with.
I think it’s more touching on the vulnerability of infrastructure if a larger portion is run by only one OS. Something a lot of usb here may realize, but the general public has never really understood it. Where a scenario like this or similar can can cause a wide-spread blackout, all from a single bug; be it from popular software, or the OS itself.
That’s not correct. Crowdstrike does also work with Mac and Linux, but this particular incident only impacted the Windows sensor.
They actually had a similar issue with the Linux sensor a couple of months ago, which… doesn’t speak well of their update process.
This is the extremely important akshually line anyway. Let’s all pretend that every OS is just as shitty because it lets us correct others on the Internet constantly
Another hivemind circlejerk
What monopolization? Crowdstrike has plenty of competitors.
So does Google. And yet they’re still being taken to court for monopolistic practices by the U.S. Department of Justice.
Monopoly doesn’t mean literally no competitors in the real world. It means no competitors worth noting because everyone has been corralled into using a single company.
Crowdstrike is big, but not that big.
About half of my clients use them; and of those, about a third are halfway through ripping them out in favor of MS defender.
(MS is definitely “that big”)
It means no competitors worth noting because everyone has been corralled into using a single company.
Right. That’s not the case here. Crowdstrike competes with a dozen other EDR products. Using the number of ratings as a proxy for popularity, they’re not even the most popular.
That dozen don’t seem to add up to much together considering the massive global nature of this outage.
Because “we didn’t crash” doesn’t make the news. My company wasn’t affected, so nobody cared about us.
That’s great that your company wasn’t affected.
I hope no one was trying to fly a plane to get to your company yesterday.
Insert Nicholas Cage “You don’t say?” meme here
It’s a shitty state of affairs but I bet nothing’s gonna change.
The title reminds me of the microsoft monopoly issue ages ago. They had to bail ou Apple so ms could have competition.
