Global internet outage - Disruption to airlines, banks, media and other businesses worldwide - July 19, 2024

[Dotta]

https://twitter.com/x/status/1814294156044275864

https://twitter.com/x/status/1814283925369290797

 

[Eli9]

Japan mostly spared in global IT meltdown despite reports of some bugs

Around the world, airlines, network broadcasts and banks said their systems were disrupted.

www.japantimes.co.jp

 

[imstilla.grandma]

A Canadian technology analyst says the incident shows the nearly "universal" use of Windows products for key digital infrastructure highlights how quickly security issues can start to cascade.

"It's so universal, so common, the impact is much wider than would normally be the case." Carmi Levy, a Canadian technology expert, told CTV Your Morning's Anne-Marie Mediwake on Friday. "We're talking about companies all around the world."

The software detects and blocks hacking threats, but like other cybersecurity products, it requires deep-level access to a computer’s operating system

"The irony is they're a cyber security company… they certainly didn't provide a lot of security here," says Levy.

What we know about the reasons behind global internet outage

A Canadian technology analyst says a failed update from a key cybersecurity provider shows the nearly "universal" use of Windows products for key digital infrastructure and highlights how quickly security issues can start to cascade.

www.ctvnews.ca

 

[imstilla.grandma]

Passengers queue at Gatwick Airport during an outage in Crawley, England

The global cyber outage that caused disruptions to airlines, businesses and emergency services on Friday could be the “largest IT outage in history,” according to cybersecurity expert Troy Hunt.

“I don’t think it’s too early to call it: this will be the largest IT outage in history,” Hunt said in a post on X on Friday.

“This is basically what we were all worried about with Y2K, except it’s actually happened this time,” he added.

Other recent large outages:

• It remains to be seen whether the scale and impact of the outage will reach the level of the “NotPetya” cyberattack in 2017, which the Kremlin was widely considered to have been behind. The 2017 attack initially targeted computers in Ukraine but affected companies globally. The White House upbraided Russia for “the most destructive and costly cyber-attack in history.”
• There was another mass internet outage in 2021 across the world, following a bad software update by Fastly, a company which runs a content delivery network of servers and data centers.
• In January 2023, a computer outage at the US Federal Aviation Administration caused thousands of canceled or delayed flights but did not directly affect other sectors. The FAA said at the time that they had traced the fault “to a damaged database file” and that there was “no evidence of a cyberattack.”
• Last month, a systems outage at software provider CDK Global, which the company says was caused by a cyberattack, resulted in chaos at car dealerships in the US, making it difficult for it to track customer interactions, orders and sales. Sources told CNN that CDK Global appeared to have paid a $25 million ransom to the hackers in order for the outage to end.

https://www.cnn.com/business/live-news/global-outage-intl-hnk/index.html

 

[Inthedetails]

NYC Mayor and various agencies are giving a presser right now. I'm not hearing anything dire. (Not sure how to link as I'm listening live over the radio on WNYC.)

Seems like things are getting back on track.

jmo

 

[nerosleuth]

I remember when the calendars were all about to flip to the year 2000, and the massive IT precautions and backups that had to be done in case the systems collapsed. Y2K it was called.

We put in tons of overtime hours at the place I was working, just to get ready. (Then nothing adverse happened. )

I remember Y2K very well too as I worked a 13 hour shift on Y2K at work.

Everything ran fine and nothing crashed at midnight on Y2K.

 

[Grouse]

SBM
God I hope there's no fatalities and this doesn't happen again.
This is giving me flashbacks to 2022 (my fellow Canadians will know) of a massive tech outage we had nation-wide.
We lost internet, cable, public transportation, government services, debit machines, traffic lights, fire alarms/sprinkler systems, and 911 services.. and someone did die. It took 2 days to resolve this little maintenance whoopsie... apparently they've put solutions in place finally (almost the same thing happened a year prior) but I guess we will just have to wait and see what happens when another software update goes wrong.

2022 Rogers Communications outage - Wikipedia

en.m.wikipedia.org

BBM. Having worked in IT myself I can understand how an error can be made and an update can go wrong and get out. What I find harder to understand is how it can be rolled out across the world in that state. In my day it was all about testing, and rolling updates out in controlled phases, so these things things got picked up before they affected the whole company. JMO.

 

[Inthedetails]

Okay, now it's personal. I have a family member trying to fly out of D.C. today. The flight was canceled and rebooked on a flight that goes way, way out of the way to get a connecting flight home. Many people will be having travel frustration today and I send them patience!

jmo

 

[mickey2942]

That happened to me last night at work. My work computer screen went completely blue, and crashed. Fortunately, I am not working in a control tower at LAX!!! How crazy would that have been!

It was about 10:20 MDT for me, I rebooted, and got back on in about 10 minutes.

Crazy enough, IT states it was an update from "Crowdstrike" a vendor system used to protect computer systems from "malware". LOL. To prevent stuff like this from happening

 

[Grouse]

That happened to me last night at work. My work computer screen went completely blue, and crashed. Fortunately, I am not working in a control tower at LAX!!! How crazy would that have been!

It was about 10:20 MDT for me, I rebooted, and got back on in about 10 minutes.

BBM.

Known as a BSOD - blue screen of death. Surprised a reboot fixed it - I thought there was more involved to recover each computer?

 

[imstilla.grandma]

“This is a very, very uncomfortable illustration of the fragility of the world’s core Internet infrastructure,” said Ciaran Martin, professor at Oxford University’s Blavatnik School of Government and former head of the UK National Cyber Security Centre. While the core problem appeared simple, which should make it short-lived, its immediate impact was remarkable, Martin said.

“I’m struggling to think of an outage at quite this scale.”

https://english.alarabiya.net/News/world/2024/07/19/global-internet-outage-is-affecting-airlines-banks-and-media-from-us-to-australia

 

[mickey2942]

BBM.

Known as a BSOD - blue screen of death. Surprised a reboot fixed it - I thought there was more involved to recover each computer?

Yeah. I don't know, I took a picture of my screen, mailed it to my supervisor from my phone. Logged back in, clocked out, turned off my computer. I didn't try to work after that, my shifts ends at 10:30.

 

[Inthedetails]

“This is a very, very uncomfortable illustration of the fragility of the world’s core Internet infrastructure,” said Ciaran Martin, professor at Oxford University’s Blavatnik School of Government and former head of the UK National Cyber Security Centre. While the core problem appeared simple, which should make it short-lived, its immediate impact was remarkable, Martin said.

“I’m struggling to think of an outage at quite this scale.”

https://english.alarabiya.net/News/world/2024/07/19/global-internet-outage-is-affecting-airlines-banks-and-media-from-us-to-australia

It's a bit re-assuring how fast everything seems to be recovering, though. What I'm hearing now is not about outages but how things are getting back to functioning mode.

 

[orchidaceaes]

BBM.

Known as a BSOD - blue screen of death. Surprised a reboot fixed it - I thought there was more involved to recover each computer?

A similar thing happened at my workplace today. IT recommended that we reboot and that should fix the issue. It worked for me but not my in-office coworker. Also didn't work for the handful people who keep hitting "reply-all" to the IT email.

 

[JDG]

SBM
God I hope there's no fatalities and this doesn't happen again.
This is giving me flashbacks to 2022 (my fellow Canadians will know) of a massive tech outage we had nation-wide.
We lost internet, cable, public transportation, government services, debit machines, traffic lights, fire alarms/sprinkler systems, and 911 services.. and someone did die. It took 2 days to resolve this little maintenance whoopsie... apparently they've put solutions in place finally (almost the same thing happened a year prior) but I guess we will just have to wait and see what happens when another software update goes wrong.

2022 Rogers Communications outage - Wikipedia

en.m.wikipedia.org

IIRC, it was Rogers customers who lost access.

We use Telus and Teksavvy for internet so were good to go.

At the time, we also had a land line with Bell so still had emergency services.

Bell has since removed all copper wired phones, so emergency services would be lost.

Many people I know were convinced to have two phones, each with different providers, after this fiasco.

 

[Grouse]

A similar thing happened at my workplace today. IT recommended that we reboot and that should fix the issue. It worked for me but not my in-office coworker. Also didn't work for the handful people who keep hitting "reply-all" to the IT email.

Oh no! The "reply to all" on emails to large groups - I'd forgotten how often I had to deal with the fallout from that when I was working. If it had a large attachment it could bring an email system to it's knees. We had to put in a restriction on the size of emails in the end - some management weren't happy about that but then they were less happy if email was down.

 

[vls12345]

Flight cancelations at 2k in the US.

https://www.cnn.com/business/live-news/global-outage-intl-hnk/index.html

"Delta has canceled at least 640 flights, and Hartsfield-Jackson Atlanta International Airport currently has the most cancellations in the US with 226."

Not a surprise it's Atlanta as it's the world's busiest airport. (And Delta is the main carrier there.) Must be chaos there today.

 

[Inthedetails]

Flight cancelations at 2k in the US.

https://www.cnn.com/business/live-news/global-outage-intl-hnk/index.html

"Delta has canceled at least 640 flights, and Hartsfield-Jackson Atlanta International Airport currently has the most cancellations in the US with 226."

Not a surprise it's Atlanta as it's the world's busiest airport. (And Delta is the main carrier there.) Must be chaos there today.

That's where my family member needs to get. It's nuts. No one-way rental cars available from the major city where she currently is stuck so no option but to keep waiting for a flight.

It's a mess, but it will straighten out eventually. Always travel with at least clean undies and a toothbrush in your carry-on!

jmo

 

[cujenn81]

A lot of people thought the Y2k thing was a hoax, when in fact it was very serious and could have led to what happened today, except for those dedicated IT people who worked round the clock to prevent the worst from happening.

They potentially saved peoples lives.

A Guardian article from 2019

The millennium bug was real – and 20 years later we face the same threats | Martyn Thomas

We fixed Y2K, but we’re fools to be complacent about the vulnerability of computer systems, says IT professor Martyn Thomas

www.theguardian.com

This! My father-in-law worked for the U.S. State Department at the time — specifically, the Office of Intelligence Research — and they put in A LOT of work creating safeguards to prevent a Y2K meltdown and national security disaster. If the world had just sat back and done nothing to prepare for it, there would have most certainly been a massive global systems failure.

 

[cujenn81]

It’s disturbing to me how many businesses and government agencies have no contingency plan for system failures like this. That’s precisely why I think it’s foolish for so many businesses to put all their eggs in the AI basket. Our society is entirely too reliant on computers these days, and it scares me to think about how easily we could be crippled by right cyberattack. Especially when it comes to services like 911 and healthcare. Health and security services really should be required to have a Plan B in place for computer failures. Honestly, it’s appalling to me that they don’t already.