Cloud solutions exclusive to our partnership.
Apple software tools to boost your productivity.
Secure collaboration with our Microsoft 365 suite.
Unmatched cloud security via our partnership.
Cloud computing managed by certified experts.
Network management tools for businesses.
The recent CrowdStrike update has led to persistent outages, yet the company has assured that the problem isn’t due to a security incident or a cyberattack.
Instead, a software defect within CrowdStrike’s channel file was identified as the root cause. Engineers worked diligently to isolate the issue and deployed an update to rectify the situation.
This incident highlights the intricacies involved in software management and the potential for unintended consequences even in well-established systems. The deployment of these updates has shown the need for rigorous testing and robust mechanisms to manage outages. It serves as a reminder of the complex nature of software maintenance and the importance of swift response strategies.
While this CrowdStrike update aimed at enhancing system functionality, it resulted in a significant outage instead—a lesson in how critical software updates can be. As engineers continue to monitor and manage the situation, customers can expect ongoing updates to ensure full resolution.
Systems have been heavily impacted since Friday, and the situation persists even today. Among the most affected are businesses and crucial infrastructure in the USA, including government agencies. The persistence of these outages is causing widespread difficulties; businesses are struggling to maintain operations, and various government departments are unable to perform essential functions.
Critical Infrastructure: 8.5 million devices used by 70% of Fortune 500 companies for banking, power/water supply, hospitals, and airports.
Flight Cancellations and Delays: On Sunday, more than 1,800 US flights were canceled and over 9,900 delayed.
Hotel Impacts: Major hotels, including Marriott International and some Hiltons, experienced payment processing and check-in delays.
On July 19, 2024, CrowdStrike released an update to their security software for Windows computers. This update was meant to improve protection, but it accidentally caused many computers to crash and show a blue screen.
The problem started around 04:09 UTC and was fixed by 05:27 UTC the same day. It’s important to know that this issue was not caused by a cyberattack.
The update affected a specific file used by CrowdStrike’s software, which helps manage communication between different parts of the computer. This file, named “C-00000291-.sys,” played a role in how the software handled certain types of data. The update introduced a mistake that led to the crashes.
However, the problem isn’t just affecting a few users; it’s widespread, causing disruptions across different systems. Numerous users reported that their systems experienced interruptions, making it clear that this isn’t a minor issue but a significant cybersecurity concern. The CrowdStrike glitch extended to critical systems, including hospitals and 911 emergency response units, further highlighting the urgency of resolving this issue promptly.
The update that caused the failure for CrowdStrike went to every version of the driver because it was a definition update, not an actual sensor or driver update. Most well-run IT organizations implement a phased rollout approach. This means they release updates in waves, starting with a small percentage of users, then gradually expanding the rollout if no major issues are detected. This way, if a problem arises, it can be quickly addressed before it affects a large number of users.
However, CrowdStrike’s deployment of definition updates seems to ignore this phased approach completely. As a result, the glitch impacted a vast number of users simultaneously. Since Friday, both technical teams and end-users have been scrambling to mitigate the bad impacts of this unanticipated glitch. By the time the bug was identified in their update, a considerable number of systems had already been compromised. Users and stakeholders in the cybersecurity community are questioning how such an error, especially from a renowned name like CrowdStrike, could have occurred in the first place.
The CrowdStrike outage affected numerous industries, including emergency communications, government services, health care systems, major airlines, and various businesses. Key impacted organizations included large hospital systems, 911 emergency response units, government agencies, and major airlines such as Delta, American Airlines, and United Airlines.
The outage was caused by a software defect within CrowdStrike’s sensor configuration update, which triggered a logic error leading to system crashes and blue screens (BSOD) on impacted systems. This issue was not the result of a cyberattack but rather a problem with the update itself.
The outage led to significant disruptions across multiple sectors:
A definition update, like the one that caused the CrowdStrike outage, updates the database of known threats and rules for detection, while a sensor or driver update involves changes to the actual software components that interact directly with the operating system.
Partnering with a reliable managed IT services provider like IT GOAT can help protect your business. IT GOAT provides comprehensive support, including business continuity planning and managed detection and response (MDR) services, to enhance your cybersecurity measures and ensure system stability.
Major hotels, including Marriott International and some Hiltons, were impacted both in payment processing and check-in delays. US-based airlines like American, United, Delta, Allegiant Air, Sun Country, and Frontier reported issues on Friday. Delta paused all its flights Friday morning while its systems were offline; more than 3,500 Delta and Delta Connection flights were canceled through Saturday. Experts urge travelers forced to cancel trips to explore refund options, as new federal rules guarantee cash refunds, not vouchers or travel credits.
Airline Issues:
The widespread usage of CrowdStrike’s software in the USA led to an outage that penetrated numerous industries, including emergency communications, government services, and health care systems. On July 20, 2024, travelers at Hartsfield-Jackson International Airport in Atlanta, Georgia, experienced significant delays. Large hospital systems like Mass General Brigham in Massachusetts, Penn Medicine in Pennsylvania, and Mount Sinai Health System in New York reported being affected by the outage. Emory Healthcare in Atlanta said procedures at ambulatory surgical centers and hospitals were delayed until systems stabilized.
Healthcare Impact: Major hospital systems like Mass General Brigham, Penn Medicine, and Mount Sinai Health System experienced delays in procedures. Cancer centers such as Dana-Farber Cancer Institute and Memorial Sloan Kettering Cancer Center paused certain procedures and scheduled appointments.
Emergency Services Disruption: In a few areas, including Arizona and Alaska, 911 services were briefly disrupted before being restored.
Government Services and Transportation: Social Security offices, local Department of Motor Vehicles offices, and public transportation systems in Washington, DC, and Pennsylvania temporarily paused operations but eventually restored by Friday.
The impact of the CrowdStrike glitch, which initially seemed concentrated, has shown just how interconnected our systems are and how any weakness can cascade into widespread issues. This incident paints a stark picture of our vulnerability and highlights the necessity of having a robust, multi-layered approach to cybersecurity. Today’s crisis is a wake-up call that can’t be ignored; it’s clear that significant improvements are needed to prevent such catastrophic failures in the future.
Endpoint protection systems consist of two main components: a backend control center and agent software installed on endpoints, which include mobile devices, computers, servers, and network appliances. The persistent issue with such endpoint protection software is that it needs to integrate deeply into the operating system, often bypassing the security measures of the system itself, such as in Windows.
The agent software continuously runs on the endpoints and comprises a core application, including a user interface (GUI), and “sensors” that hook into the operating system to intercept processes and executions. For example, if you try to launch a program on Windows, the agent software’s sensor is alerted by the operating system. It then checks the executable and, if necessary, prevents it from running. The sensor informs the main application of the blocked execution, which in turn notifies the control center over the Internet.
Core Components: Endpoint protection systems include a backend control center and agent software on devices like computers and servers, integrating deeply into the operating system.
Functionality: The agent software continuously monitors the system, intercepting processes and executions, and preventing potentially harmful actions.
Implications of Outages: Disruptions in endpoint protection software like CrowdStrike Falcon can have widespread implications, affecting various organizations, including hospitals and government agencies, highlighting the essential role of these systems in maintaining security.
The CrowdStrike outage underscores the critical importance of having a managed IT services provider like IT GOAT that implements updates on a carefully maintained scale to ensure system stability. By doing so, you can minimize the risk of widespread disruptions and maintain the integrity of your operations. For more information on how IT GOAT’s managed IT services can support your business continuity and enhance your cybersecurity measures, check out our Data Backup & Disaster Recovery and Managed Detection and Response (MDR) pages to learn more about these essential services.
See the power of IT GOAT.
The world’s most advanced cybersecurity platform catered specifically to your business’ needs.
Keep up to date with our digest of trends & articles.
By subscribing, I agree to the use of my personal data in accordance with IT GOAT Privacy Policy. IT GOAT will not sell, trade, lease, or rent your personal data to third parties.
Mitigate All Types of Cyber Threats
Experience the full capabilities of our advanced cybersecurity platform through a scheduled demonstration. Discover how it can effectively protect your organization from cyber threats.
IT GOAT: Threat Intel & Cyber Analysis
We are experts in the field of cybersecurity, specializing in the identification and mitigation of advanced persistent threats, malware, and exploit development across all platforms.
Protect Your Business & Operations
Exceptional performance in the latest evaluations, achieving 100% prevention rate and providing comprehensive analytic coverage, unmatched visibility, and near-instant detection of threats.