CrowdStrike-Microsoft Outage: 5 Key Updates To Know

Flight cancellations continued through the weekend while the two vendors sought to help expedite the recovery process for the millions of affected Windows devices.

Following the massive CrowdStrike-Microsoft outage early Friday morning, flight cancellations continued through the weekend while the two vendors sought to help expedite the recovery process for the millions of affected Windows devices.

CrowdStrike’s disastrous Falcon update led to the “blue screen of death” for Windows systems worldwide on Friday and hobbled much of what the modern world depends on, from air travel to health care to banking and beyond. Experts have called it the largest IT outage of all time.

[Related: CrowdStrike Has Been Doing Updates This Way ‘For Many Years’: What Went Wrong?]

“We understand the gravity of this situation and are deeply sorry for the inconvenience and disruption,” CrowdStrike said on its page for the outage incident. “Our team is fully mobilized to ensure the security and stability of CrowdStrike customers.”

Microsoft disclosed Saturday that 8.5 million Windows devices were impacted by CrowdStrike’s update, amounting to less than 1 percent of Windows systems.

“While the percentage was small, the broad economic and societal impacts reflect the use of CrowdStrike by enterprises that run many critical services,” Microsoft said.

What follows are five key updates to know on the CrowdStrike-Microsoft outage.

Outage Woes Continue For Some

Following the outage early Friday, “customers did resolve the vast majority of these issues over the weekend”—although there are “some lingering issues into this week so far out of the gates,” Wedbush Securities’ Daniel Ives wrote in a note to investors Monday.

CRN has reached out to CrowdStrike for comment.

“We are seeing IT outage issues still persist in spots around the globe from the CrowdStrike IT outage heard around the world on Friday,” wrote Ives, managing director and senior equity research analyst at Wedbush. “This is not good news for George Kurtz & Co. in an already bad situation as it appears a number of businesses are still finding difficulty on the path to normalization from this IT outage despite fixes/mediations released throughout the weekend.”

Delta Continues To Struggle

As one of the hardest-hit airlines from the get-go, Delta has reportedly also been having the most difficulty with returning to normal operations after the outage. In an update posted Sunday afternoon, Delta disclosed that more than 3,500 flights had been canceled Friday and Saturday.

“Cancellations continue on Sunday as Delta’s teams work to recover our systems and restore our operation,” Delta said in the update. “Canceling a flight is always a last resort and something we don’t take lightly.”

Citing the flight tracking site FlightAware, Quartz reported that 1,300 flights had been canceled Sunday and more than 600 flights were canceled as of Monday morning.

CRN has reached out to Delta for comment.

United also reportedly continued to face difficulties over the weekend but to a lesser extent than Delta. American Airlines had reportedly fully recovered as of Saturday afternoon.

CrowdStrike ‘Remediation And Guidance Hub’

Over the weekend, CrowdStrike posted a “Remediation and Guidance Hub” that aims to assist with recovering from the outage.

Sections on the site include technical details as well as important areas for IT pros to focus on —such as how to identify impacted hosts, recover Bitlocker keys, remediate the impacted hosts and recover cloud-based environments.

In addition, CrowdStrike said on the page that it has “tested a new technique to accelerate impacted system remediation.”

“We’re in the process of operationalizing an opt-in to this technique,” the company said. “Customers are encouraged to follow the Tech Alerts for latest updates as they happen, and they will be notified when action is needed.”

Microsoft Launches Recovery Tool

Microsoft Sunday released a free tool to help clients recover from the outage, enabling IT administrators to more quickly recover from the blue screen of death using a more-automated approach.

Instead of trying to simply turn the machines off and on several times if a customer is running virtual machines inside Azure, Microsoft is offering two main repair options to “help IT admins expedite the repair process,” Microsoft said in a post update Sunday.

‘Logic Error’ To Blame

In an update late Friday evening, CrowdStrike identified a “logic error” as the culprit in the Microsoft outage. The programming error was triggered by a sensor configuration update to Falcon.

Such updates “are a normal part of the sensor’s operation and occur several times a day in response to novel tactics, techniques and procedures discovered by CrowdStrike,” the company said in the post.

The sensor configuration update that ultimately triggered the logic error was released to Windows systems shortly after midnight EST on Friday, the company said in the post.

For a still-unknown reason, “this configuration update triggered a logic error resulting in a system crash and blue screen (BSOD) on impacted systems,” the company said.