Start your day with intelligence. Get The OODA Daily Pulse.
In the spirit of the significance of tracking the global impact of disruptive events and encouraging the sharing of relevant stories for compilation, the following is our latest tracking of the Crowdstrike Incident since our last update on 7/22 – The Crowdstrike/Microsoft Global IT Outage Debacle: Ongoing Impacts and Recent Updates and the July 2024 OODA Network Monthly Meeting: A Real-time Discussion of the Crowdstrike Global IT Outage.
“Sadly, it was an interesting lesson for the bad guys. [They learned] It was one mechanism that started the entire process.”
Who loves a global outage? Phishers, fraudsters and all manner of creeps
Well that was fast. Criminals didn’t waste any time taking advantage of the CrowdStrike-Microsoft chaos and quickly got to work phishing organizations and spinning up malicious domains purporting to be fixes. Just hours after a faulty CrowdStrike file shut down Windows machines around the globe, reports surfaced of scam emails using the outage as a lure and otherwise trying to use the massive outage as a means to pursue criminal activities.
“Some reports we have seen indicate that there may be phishing emails circulating claiming to come from ‘CrowdStrike Support’ or “CrowdStrike Security,” said Johannes Ullrich, dean of research for SANS Technology Institute and the founder of the Internet Storm Center. While he did not have any samples to share at the time, “attackers are likely leveraging the heavy media attention,” Ullrich added. “Please be careful with any ‘patches’ that may be delivered this way.”m ICS also listed one domain that is “possibly” linked to these phishing attacks:
crowdfalcon-immed-update [ .] com
Other phony domains posing as fixing sites surfaced on social media, with security researchers warning users not to pay for a fix — there’s free support from the real CrowdStrike — as some of the fraudulent websites asked for bitcoin and PayPal “donations.” Additionally, while CrowdStrike CEO George Kurtz, in a statement on X, assured customers “this is not a security incident or cyberattack,” the software flaw does make it that much easier for network intruders to sneak in while system admins work to implement the fix.
null pointers strike again!
— Evis Drenova (@evisdrenova) July 19, 2024
looks like the crowdstrike issue is a dereferenced null pointer that’s trying to access null memory using the ‘mov r9d, dward ptr r[8]’ assembly operation.
brutal. pic.twitter.com/cQnLv1SyRm
This strange tweet got >25k retweets. The author sounds confident, and he uses lots of hex and jargon. There are red flags though… like what’s up with the DEI stuff, and who says “stack trace dump”? Let’s take a closer look… 1/n pic.twitter.com/Lt4VfCs15s
— Tavis Ormandy (@taviso) July 20, 2024
This document provides an executive summary of the findings of CrowdStrike’s Root Cause Analysis (RCA) report. The full report elaborates on the information previously shared in our preliminary Post Incident Review (PIR), providing further depth on the findings, mitigations, technical details and root cause analysis of the incident.
Download the Root Cause Analysis PDF
Introduction
CrowdStrike was founded with a mission to protect customers against today’s adversaries and stop breaches. On July 19, 2024, as part of regular operations, CrowdStrike released a content configuration update (via channel files) for the Windows sensor that resulted in a system crash. We apologize unreservedly.
We acknowledge the incredible round-the-clock efforts of our customers and partners who, working alongside our teams, mobilized immediately to restore systems and bring many back online within hours. As of July 29, 2024, at 8:00 p.m. EDT, ~99% of Windows sensors were online, compared to before the content update. We typically see a variance of ~1% week-over-week in sensor connections. To any customers still affected, please know we will not rest until all systems are restored.
What Happened
The CrowdStrike Falcon sensor delivers AI and machine learning to protect customer systems by identifying and remediating the latest advanced threats. In February 2024, CrowdStrike introduced a new sensor capability to enable visibility into possible novel attack techniques that may abuse certain Windows mechanisms. This capability pre-defined a set of fields for Rapid Response Content to gather data. As outlined in the RCA, this new sensor capability was developed and tested according to our standard software development processes.
On March 5, 2024, following a successful stress test, the first Rapid Response Content for Channel File 291 was released to production as part of a content configuration update, with three additional Rapid Response updates deployed between April 8, 2024 and April 24, 2024. These performed as expected in production.
On July 19, 2024, a Rapid Response Content update was delivered to certain Windows hosts, evolving the new capability first released in February 2024. The sensor expected 20 input fields, while the update provided 21 input fields. In this instance, the mismatch resulted in an out-of-bounds memory read, causing a system crash. Our analysis, together with a third-party review, confirmed this bug is not exploitable by a threat actor.
While this scenario with Channel File 291 is now incapable of recurring, it informs the process improvements and mitigation steps that CrowdStrike is deploying to ensure further enhanced resilience.
See also:
Researchers track the healthcare sector as experiencing the biggest financial losses, with banking and transportation following close behind.
As the CrowdStrike Falcon outage story continues to unfold, the monetary losses to businesses from the global incident continue to rise: The volume is likely to reach $5.4 billion in costs for Fortune 500 companies, according to a report from Parametrix. Parametrix researchers have found that roughly 25% of Fortune 500 companies experienced disruptions due to the incident, the most heavily affected industries financially being healthcare ($1.94 billion in estimated losses) and banking ($1.15 billion). In addition, a shocking 100% of the transportation and airlines sector was affected, and the group will rack up an estimated $0.86 billion in losses, according to the forecast. The $5.4 billion estimate excludes Microsoft. The researchers noted that the outage impact to some industries, like software and IT-related services, is more likely to cause a “ripple effect beyond Fortune 500 companies,” though hard numbers were not quantified in the report.
There are a number of US Securities and Exchange Commission (SEC) reporting implications arising from the server-related outages caused by CrowdStrike’s defective software update on July 19, 2024, and their impacts on public companies, particularly in light of the SEC’s new cybersecurity disclosure rules. While the situation on the ground – as well as answers to these questions – is still very much evolving, public companies impacted by the CrowdStrike update should consider doing the following:
CISA director calls CrowdStrike-linked outage a “dress rehearsal” for what China may have planned for U.S. critical infrastructure.
The faulty CrowdStrike Falcon update that caused millions of computers around the world to malfunction was “a useful exercise” for understanding what Chinese-linked cyber operations focused on sensitive U.S. networks could accomplish, a top U.S. cybersecurity official said Wednesday. Jen Easterly, the director of the Cybersecurity and Infrastructure Security Agency, told a large crowd at the annual Black Hat cybersecurity conference that the fallout of the CrowdStrike situation — which disrupted medical care, canceled flights and shuttered retailers — showed what effects Chinese-linked activity tracked as Volt Typhoon could generate.
“What was going through my mind was that, oh, this is exactly what China wants to do, but without rolling back the updates such that we could all reboot our systems,” Easterly said during a keynote address alongside top cybersecurity officials from the U.K. and Europe. Volt Typhoon is the Microsoft-given name for suspected Chinese cyber activity targeting critical infrastructure organizations in the United States. Officials from the U.S. and other western countries have, for more than a year, warned that the Chinese-linked group aims to pre-position cyber capabilities in key networks to be able to disrupt operations in the event of military conflict or crisis involving China.
The malicious activity comes as CrowdStrike customers continue to recover from the July 18 outage.
Five days after a faulty update to CrowdStrike’s Falcon security software hobbled millions of Windows computers around the world, cybercriminals and hacktivist personas are taking advantage of the situation with newly registered domains, malware attached to files with CrowdStrike-themed names and at least one apparent instance of a data wiper. CrowdStrike has documented multiple instances of likely criminal activity tied to the incident, including a Word document laced with the Daolpu information stealer and a zip file targeting Latin American-based CrowdStrike customers with the HijackLoader malware, which is typically used to deliver other malware packages, and a Python-based information stealer tracked as “Connecio.”
Additionally, a phishing email with a PDF purporting to explain how to remediate last week’s Falcon issue delivered a zip file laced with wiper malware, according to sandbox company ANY.RUN, which called it one of the most “sophisticated” outage-related attacks thus far. “Handala Hack,” a pro-Palestinian hacktivist persona known for attacking Israeli targets, claimed responsibility for the wiper attack mentioned by ANY.RUN. In a June 21 Telegram post, they asserted — without providing evidence — that they had targeted “thousands of Zionist organizations!”
Enrique Hernandez, threat research director at Splunk, said in a Tuesday post on X that he identified more than 2,000 CrowdStrike-related domains registered in the past seven days. An analysis of the top 25 suggests that “most of them are looking pretty funky,” Hernandez wrote. James Spiteri, a director of product management with Elastic, wrote in a LinkedIn post Sunday that he had documented more than 141 certificates generated for what looks “like (mostly) bogus [CrowdStrike] domains. Hope this list helps folks keep a lookout for any phishing.” The list had grown to 193 by mid-afternoon Tuesday.
With 36 hours of perspective, and readily acknowledging there is still much we need to learn about the event, I wanted to provide some personal thoughts on yesterday’s massive IT outage. While this was a technology incident, not a cyber-attack, in our role as National Coordinator for critical infrastructure security and resilience, CISA worked aggressively with Crowdstrike and partners across government and industry at all levels to understand the breadth of impacts to critical infrastructure and help drive remediation and risk mitigation. (Read More)
The global impact of the flubbed CrowdStrike update and ensuing Microsoft outage was a wakeup call for European and U.S. cybersecurity leaders. The topic took center stage here at the Black Hat USA 2024 opening keynote. Open questions included: How could a single vendor trigger such massive global disruptions, what does this portend for vital systems of democracy such as elections and how can the cybersecurity community ensure it doesn’t happen again? “Sadly, it was an interesting lesson for the bad guys. [They learned] It was one mechanism that started the entire process,” said Hans de Vries, COO of the European Union Agency for Cybersecurity, commenting on the CrowdStrike bungled software update.
Joining de Vries on stage was Jen Easterly, director of the U.S. Cybersecurity and Infrastructure Security Agency, and Felicity Oswald OBE, CEO of the UK’s National Cyber Security Centre. Easterly warned there has been a lot of “irresponsible noise on the CrowdStrike incident” — however, the gravity of the outage can’t be ignored. She outlined her three top takeaways from the CrowdStrike outage. (Read More
“This is Not a Security Incident or Cyberattack”: Microsoft and Crowdstrike Scramble to Patch ‘Largest IT Outage in History’: At approximately 3 AM EST on July 19th, reports started crossing the transom of a global IT outage impacting a broad range of industries, causing airlines, banks, media broadcasters, and shipping lines to shut down operations. Boston’s Logan Airport was shut down this morning, Washington D.C.’s Metrorail has been impacted, and planes were grounded at many airports around the world. This post is a quick and dirty tick-tock of the incident and the response from Microsoft and Crowdstrike. For CISOs in mitigation mode, we have compiled some technical links here as well.
The Crowdstrike/Microsoft Global IT Outage Debacle: Ongoing Impacts and Recent Updates: In an update of our initial post (assessing the early onset of the Global IT outage) on Friday, 7/19/24 at 10 AM, included here is CISA’s formal response on Friday at1 p.m. EST (with updates from CISA through 7/21), an interesting quick take from Beijing on “why China was largely unaffected by Friday’s IT outage”, amongst other ongoing impacts and updates from CNBC, Wired, and Interos.
The Botched Update Heard Around the World Calls for More Diversification: Recently, a major IT outage impacted the global community. In the midst of global geopolitical conflicts and states waging war on one another, the major disruption was not the result of hostile state cyber assets or enterprising nonstate hacktivist enclaves engaged in digital struggle. This unprecedented glitch was caused by a faulty software update from CrowdStrike, a trusted U.S. cybersecurity vendor that has been a leader in the cybersecurity space for several years. Per the company’s statement, a defect in a content update to its software for Windows hosts caused the disruption. As a consequence, the company’s global customers across government services, emergency call centers, airlines, hospitals, and banks, to name just a few, found themselves in a predicament, facing Microsoft’s dreaded “blue screen of death.” In the United States alone, nearly 3,000 flights have been delayed or canceled, and some organizations are still trying to recover from the fallout.
https://oodaloop.com/archive/2024/07/31/the-july-2024-ooda-network-monthly-meetings-a-real-time-discussion-of-the-crowdstrike-global-it-outage/