Crowdstrike’s Big Strikeout: How a Security Company Nearly Turned Off the Internet
On Friday, July 19th, flights all around the world were grounded, banks closed, self-checkouts at supermarkets crashed, ATMs failed, and government agencies and major companies had to close their doors while IT workers scrambled to fix entire buildings full of crashed computers. It’s one of the largest cyber outages of all time, and it all happened because just 8.5 million Windows machines, a tiny fraction of the total fleet of Windows machines, all crashed at once. This began a domino effect that rippled through the internet and society, reports Risky.biz.
How did it happen? Crowdstrike is a security company that caters to important corporate clients with critical networks. Their software does what’s called endpoint detection and response, or EDR, which you could think of as a human-managed version of anti-virus scanning that also looks for other kinds of malicious activity. It’s powerful software designed to help security professionals protect important networks. Crowdstrike was well regarded until Friday July 19th, when they took their eye off the ball in possibly the most spectacular fashion in the history of the internet.
Crowdstrike sent out an update to their software that caused every Windows machine running that software to enter an endless cycle of crashing and rebooting. Since Crowdstrike caters to important clients, the 8.5 million machines running their software included some in critical positions. Fixing those crashed computers required manually rebooting into safe mode and removing one file. This meant that 8.5 million machines had to be manually serviced by a technician. That work was mostly completed over the weekend.
An update causing this level of crash is extremely unusual. Many expected and normal update procedures seem to have been skipped by Crowdstrike, such as any testing of the update, or phasing the rollout of the update to go to a small selection of users at a time, either of which would have prevented or mitigated the catastrophe.
The Bottom Line: We salute the thousands of IT workers who heroically sacrificed their weekends to get the world back up and running. While this outage inconvenienced millions of people, the final effects could have been much worse. For the average tech consumer there isn’t anything you need to do, nor is there anything you could have done except, perhaps, enjoy a good book for a day or two while your banks and grocery stores recovered. If you are the sort of person who writes updates to software with kernel privileges, then this is a thorough reminder of the importance of the testing you are no doubt already doing before every single release.
AT&T Admits It Lost Data on 110 Million Customers, Latest in Snowflake Fiasco That Will Never End
The telecommunications giant AT&T will be notifying nearly every one of its 110 million customers that their personal data may have been stolen in a massive security breach, reports Data Breach Today. The affected data includes call and messaging metadata such as who was called, for how long, when, and sometimes which cell towers were used. This sort of data could help scammers identify individual targets including their location, movements, and social networks.
The data was stolen from the data warehousing company Snowflake. As we reported last month, since at least June 10th, hackers have been targeting Snowflake. At least 165 Snowflake customers have had data stolen, including the US events company Ticketmaster, which may have lost data relating to 560 million customers, as reported by Wired; US car part retailer Advance Auto Parts, with 2.3 million customers affected, reported Security Week; and the financial firm Santander, which lost account information for 30 million customers and former employees, reported the BBC.
Yet even though these are breaches of Snowflake accounts, it seems the fault lies not entirely with Snowflake. In most cases, the hackers simply logged into the Snowflake accounts using the correct password, and the vast majority of the compromised accounts were not employing multi-factor authentication, reports Google’s Mandiant division, who are working with Snowflake. The hackers’ real coup was somehow installing infostealer malware on an AT&T employee’s computer, then stealing their password to access AT&T’s Snowflake account. The same likely happened to Ticketmaster, and so many others. On its own, a copy of the password should not grant access to a critical account, because critical accounts should be protected by multiple factors, but in most of these cases, multi-factor authentication was not enabled.
The Bottom Line: Use multi-factor authentication. This entire debacle could have been avoided if that simple step were taken for each of these critical accounts. Since huge amounts of personal data have been compromised, it’s important to remain vigilant against scammers who will come armed with personal information about you. We recommend using credit monitoring services through your bank and considering freezing your credit until next time you need it. Never give personal information to unsolicited callers.
Authy Leaks Millions of Phone Numbers
Authy is a popular multi-factor authentication app similar to Google Authenticator or Microsoft Authenticator. However, a recent hack has left us wondering just how secure Authy is. According to Bleeping Computer, our old friends ShinyHunters (who you may remember from last month’s newsletter as being responsible for the Ticketmaster hack) seem to have taken advantage of an unsecured Application Programming Interface endpoint within the app. ShinyHunters claims to have 33 million phone numbers that it acquired through this vulnerability. Thankfully, the attack only stole account IDs and phone numbers, so your passwords and other important data are safe. An exposed phone number is not, on its own, all that dangerous—phone books still exist, after all—but such large lists of phone numbers can be combined with other information available from data brokers to figure out who to scam and how to approach them.
The Bottom Line: If you use Authy, consider switching to a different authenticator app. Bitwarden and iCloud Keychain both offer secure ways to generate 2FA codes. Additionally, be on the lookout for any suspicious text messages such as those requesting 2FA codes or other personal information like passwords, soliciting political donations, and suggesting you’ve lost a package.
Unreported OpenAI Hack Raises Questions
Last year, OpenAI was breached by a lone hacker, and the company has only now made the incident public. According to the New York Times, a hacker infiltrated OpenAI’s internal messaging system and was able to access “details about the design of the company’s A.I. technologies.” The company made the breach known to its employees in April 2023 but chose not to disclose the incident to the public or law enforcement agencies like the FBI. The company’s executives reasoned that because they believed the hacker acted alone, there was no threat to national security. Additionally, no customer information was stolen, so the executives did not feel the need to inform OpenAI users.
Leopold Aschenbrenner, an OpenAI technical program manager, raised concerns about the company's security, but the company did not agree with his assessment. Aschenbrenner was let go from OpenAI, though company spokeswoman Liz Bourgeois claims his termination was unrelated to his security concerns.
In our opinion, it is quite concerning that such a security breach occurred, and OpenAI chose to keep it under wraps rather than at least inform the FBI. How can one trust the company to disclose future hacks? If anything, OpenAI’s handling of this breach makes me thankful that Apple is working to make Apple Intelligence as private as possible.
The Bottom Line: While OpenAI chose to keep this breach from the public, the breach thankfully did not include any customer information. However, we would still counsel caution about trusting OpenAI, both to keep itself secure and to report future cyberattacks.
If You Own One of These Linksys Routers, Change Your Router Password
Security researchers at Belgian consumer organization Testaankoop have discovered that two Linksys router systems, the Velop 6e Mesh system and the Velop 7 Mesh system, transmit Wi-Fi passwords and network names (SSIDs) in an insecure manner during initial setup, which would permit an attacker to read the password or to make changes, reports TechSpot. If you have set up one of these router systems and then left the password alone ever since, then it would be a good idea to use the Linksys Smart Wi-Fi web portal to change your Wi-Fi password. Using the web portal is secure, but using the app is not.
Your Wi-Fi system has two important passwords, the one for the Wi-Fi (which everybody knows about since you can’t connect without it) and the one to access the router’s administration panel, which is where you would go to change your Wi-Fi name or the password used to connect. In general, it is important to make sure your router’s administration panel has a strong password, and to change that password every now and then. Even if you don’t have a Velop 6e or 7 Mesh router system from Lynksys, you may take this as your periodic reminder to update your router’s admin password.
The Bottom Line: Users of Velop 6e and 7 mesh routers will need their admin password in order to access the admin panel and change their Wi-Fi password. When you change the Wi-Fi password, every device connected to the network will be disconnected, and you’ll have to go around and give each one the new password in order for them to connect again. So, while you’re at it, you might as well change the network name too, for added security. We recommend not using your address or any identifying information in your Wi-Fi network name, since the Wi-Fi network name is visible to everyone who can pick up the signal, and might help prowlers identify who owns a network to make scamming that victim easier.
