Global internet outage - Disruption to airlines, banks, media and other businesses worldwide - July 19, 2024

[nerosleuth]

The global internet outage is also affecting various local and state government agencies and businesses today in the Columbus, Ohio region.

How the global tech outage is affecting services in Columbus and Franklin County

 

[Grouse]

It’s disturbing to me how many businesses and government agencies have no contingency plan for system failures like this. That’s precisely why I think it’s foolish for so many businesses to put all their eggs in the AI basket. Our society is entirely too reliant on computers these days, and it scares me to think about how easily we could be crippled by right cyberattack. Especially when it comes to services like 911 and healthcare. Health and security services really should be required to have a Plan B in place for computer failures. Honestly, it’s appalling to me that they don’t already.

I completely agree with you but to be honest I think the ship has sailed on this, I think we have passed a point of no return. It's not just computers and the internet, we've come to rely totally on a lot of other infrastructure in the last century or so - radio communications, power grids, satellites etc. Another solar storm like the one in 1859 could cause untold damage.

Not going for my tin foil hat yet though, think we have to remember the massive benefits of this technology as well

 

[Dotta]

https://twitter.com/x/status/1814325260977684732

 

[Lorenathered]

I am in IT and have been dealing with this since I walked in this morning. Almost every system (500+) was at a bluescreen. No amount of rebooting fixed any of ours. We also had bitlocker on most of them which added to the headache. But just to put the recovery from this into perspective, I had to go to every system and boot them into recovery mode and delete the offending file from a DOS prompt. The reports and crowdstrike are making it sound like they handed us a life raft quickly. No they didn't. Most of the time, once a system is in a blue screen loop, there is no "pushing" a fix to it. So from what it takes to get all the systems back on line, that should give a good idea of how long it might take for everything to get back up and running. And also, give your IT people some slack today. It has been rough.

 

[Inthedetails]

I am in IT and have been dealing with this since I walked in this morning. Almost every system (500+) was at a bluescreen. No amount of rebooting fixed any of ours. We also had bitlocker on most of them which added to the headache. But just to put the recovery from this into perspective, I had to go to every system and boot them into recovery mode and delete the offending file from a DOS prompt. The reports and crowdstrike are making it sound like they handed us a life raft quickly. No they didn't. Most of the time, once a system is in a blue screen loop, there is no "pushing" a fix to it. So from what it takes to get all the systems back on line, that should give a good idea of how long it might take for everything to get back up and running. And also, give your IT people some slack today. It has been rough.

Oh, wow! I'd hand you a beer at the end of the day if I could. On behalf of everyone, thank you.

(I haven't heard "dos prompt" in ages. For most of us, that term has gone the way of "dial tone" these days.)

jmo

 

[JDG]

Just keep some cash on hand folks.

I recall a power outage where I was able to make a purchase at the grocery store because I had cash and didn’t care about my change.

Lots of positives about technology but keep the old ways available as backup.

 

[Lorenathered]

Oh, wow! I'd hand you a beer at the end of the day if I could. On behalf of everyone, thank you.

(I haven't heard "dos prompt" in ages. For most of us, that term has gone the way of "dial tone" these days.)

jmo

Thank goodness I have been in IT long enough to remember my DOS commands! If it weren't for bitlocker I could have booted them into safe mode and it would have been slightly easier to remove the file. It may be the old school PC tech in me but doing it in DOS was much faster in the long run anyway.

 

[whiterhino]

BBM.

Known as a BSOD - blue screen of death. Surprised a reboot fixed it - I thought there was more involved to recover each computer?

You can still use a computer with BSOD in "safe mode" and it can either be reset to factory mode or create a restore point. FWIW. Idk if that has anything to do with whatever happened today though?

 

[Knox]

I am in IT and have been dealing with this since I walked in this morning. Almost every system (500+) was at a bluescreen. No amount of rebooting fixed any of ours. We also had bitlocker on most of them which added to the headache. But just to put the recovery from this into perspective, I had to go to every system and boot them into recovery mode and delete the offending file from a DOS prompt. The reports and crowdstrike are making it sound like they handed us a life raft quickly. No they didn't. Most of the time, once a system is in a blue screen loop, there is no "pushing" a fix to it. So from what it takes to get all the systems back on line, that should give a good idea of how long it might take for everything to get back up and running. And also, give your IT people some slack today. It has been rough.

We are one of the big three waste companies, it took us out nationwide and Canada. We’re just now getting back online, it basically took us out at the knees. We have site operational plans for this, but it was very difficult to function in manual mode.

I feel very vulnerable right now, professionally and personally. God help us all if this type of event occurs again, but as a trageted attack with no fix in sight. Very scary thought!

 

[whiterhino]

We are one of the big three waste companies, it took us out nationwide and Canada. We’re just now getting back online, it basically took us out at the knees. We have site operational plans for this, but it was very difficult to function in manual mode.

I feel very vulnerable right now, professionally and personally. God help us all if this type of event occurs again, but as a trageted attack with no fix in sight. Very scary thought!

RBBM: This!

I saw a post recently talking about "the old world" back when we used radios and cash (not to make light of what is happening but the post was lol). Nothing was online!

 

[Knox]

RBBM: This!

I saw a post recently talking about "the old world" back when we used radios and cash (not to make light of what is happening but the post was lol). Nothing was online!

Five years ago less than 15% of our business was paid with a card. Today its 90%, no one carries cash or write checks. With no card reader, that means manually recording information, then shredding. Carbons don’t work any more because there is no raised surface (chip).

 

[Grouse]

You can still use a computer with BSOD in "safe mode" and it can either be reset to factory mode or create a restore point. FWIW. Idk if that has anything to do with whatever happened today though?

Yes but I don't think you could expect the average end user of a computer to know how to do that (and why should they, it's not their job), still requires IT intervention (or did in this case apparently).

 

[imstilla.grandma]

“The important part is how we learn from them and how we improve the resilience of our systems, so that similar issues do not happen again,” Andrew Sullivan, CEO of the nonprofit Internet Society said Friday.

“This was a failure of some systems using a specific operating system and a specific vendor’s management tools,” he said. “Unfortunately, those systems were used widely and for many functions critical to people’s daily lives.”

Meanwhile, some cybersecurity experts are warning that organizations affected by the internet outage should be alert for scammers.

“Organizations should be aware and wary of that and making sure that when they’re talking about getting this problem remediated, that they’re talking to trusted organizations,” said Gartner analyst Eric Grenier. “Attackers will definitely prey on organizations as a result of this.”

Live: Internet outage latest | Airlines, businesses, border crossings hit by global tech disruption

Businesses and governments worldwide experienced hours-long disruptions as a widespread technology outage affected services across industries.

www.seattletimes.com

 

[Grouse]

I am in IT and have been dealing with this since I walked in this morning. Almost every system (500+) was at a bluescreen. No amount of rebooting fixed any of ours. We also had bitlocker on most of them which added to the headache. But just to put the recovery from this into perspective, I had to go to every system and boot them into recovery mode and delete the offending file from a DOS prompt. The reports and crowdstrike are making it sound like they handed us a life raft quickly. No they didn't. Most of the time, once a system is in a blue screen loop, there is no "pushing" a fix to it. So from what it takes to get all the systems back on line, that should give a good idea of how long it might take for everything to get back up and running. And also, give your IT people some slack today. It has been rough.

Well done you! Very poor change management technique from Crowdstrike, they will lose a lot of customers I think - Elon Musk very clear and vocal on that.

 

[nhmemorymaker]

Oh, wow! I'd hand you a beer at the end of the day if I could. On behalf of everyone, thank you.

(I haven't heard "dos prompt" in ages. For most of us, that term has gone the way of "dial tone" these days.)

jmo

ha ha...thought the same when I read his/her DOS prompt statement.
And yes, beers all around to the "helpers "...

 

[imstilla.grandma]

At CrowdStrike, engineers described an atmosphere of confusion as the company struggled to contain the damage.

Executives urged employees not to speculate on why the mistake happened and directed them to focus on a fix for the computers that were affected, said two engineers who spoke on condition of anonymity because they were not authorized to speak publicly. Computers not connected to the cloud required a physical fix to the error introduced by CrowdStrike, they said, which could take weeks.

“One of the tricky parts of security software is it needs to have absolute privileges over your entire computer in order to do its job,” said Thomas Parenty, a cybersecurity consultant and a former U.S. National Security Agency analyst. “So if there’s something wrong with it, the consequences are vastly greater than if your spreadsheet doesn’t work.”

On Friday, the stock price of CrowdStrike, which reported $3 billion in annual revenue last year, closed down 11 percent

Chaos and Confusion: Tech Outage Causes Disruptions Worldwide

Airlines, hospitals and people’s computers were affected after CrowdStrike, a cybersecurity company, sent out a flawed software update.

www.nytimes.com

 

[Inthedetails]

RBBM: This!

I saw a post recently talking about "the old world" back when we used radios and cash (not to make light of what is happening but the post was lol). Nothing was online!

But, but, but....Websleuths is online.

I actually do use cash and listen to the radio! LOL. But I can mix and mingle with 2024 too.

jmo

 

[TTF14]

CrowdStrike is a very apt name for a company that caused this much upheaval to the masses.

 

[whiterhino]

Yes but I don't think you could expect the average end user of a computer to know how to do that (and why should they, it's not their job), still requires IT intervention (or did in this case apparently).

I'm just pointing out that it can be done. I'm not referring to the CrowdStrike workplace situation, I mean in general. As an average person I've managed to fix my own laptop which is how I know it can be done. Obviously not if you're in an office with an IT department.

 

[Grouse]

I'm just pointing out that it can be done. I'm not referring to the CrowdStrike workplace situation, I mean in general. As an average person I've managed to fix my own laptop which is how I know it can be done. Obviously not if you're in an office with an IT department.

That's really good that you've done that!