CrowdStrike effectively bricked windows, Mac and Linux today.

Windows machines won’t boot, and Mac and Linux work is abandoned because all their users are on twitter making memes.

Incredible work.

  • Klanky@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    133
    ·
    7 months ago

    I wish my Windows work machine wouldn’t boot. Everything worked fine for us. :-(

    • half coffee@lemy.lol
      link
      fedilink
      arrow-up
      26
      ·
      7 months ago

      Anecdotal, but my spouse was in surgery during the outage and it went fine, so I imagine they take precautions (like probably having a test machine for updates before they install anything on the real one, maybe)

      • Blank@lemmy.world
        link
        fedilink
        arrow-up
        37
        ·
        7 months ago

        There were no test rings for this one and it wasn’t a user controlled update. It was pushed by CS in a way that couldn’t be intercepted/tested/vetted by the consumer unless your device either doesn’t have CS installed or isn’t on an external network… or I suppose you could block CS connections at the firewall. 🤷‍♂️

      • Zacryon@feddit.org
        link
        fedilink
        arrow-up
        8
        ·
        7 months ago

        Depending on the machine, I guess it’s likely that those aren’t using Windoofs at all. I would be surprised if there were devices in use during surgery who run on that.

      • Dran@lemmy.world
        link
        fedilink
        arrow-up
        4
        ·
        7 months ago

        I’ve heard anecdotally that some 911 services were down in my area, but I can’t speak to how wide that was.

    • tiredofsametab@kbin.run
      link
      fedilink
      arrow-up
      9
      ·
      7 months ago

      Good News! Unless something has changed since I worked in healthcare IT, those systems are far too old to be impacted!

      I’m half-joking. I don’t know what that kind of equipment runs, but I would guess something embedded. The nuke-med stuff was mostly linux and various lab analyzers were also something embedded though they interface with all sorts of things (which can very well be windows). Pharmaceutical dispensers ran various linux-like OS’s (though I couldn’t even tell you the names anymore). Some medical records stuff was also proprietary, but Windows was replacing most of it near the end of my time.

      One place we had ran their keycard system all on a windows 3.1 box still. I don’t doubt some modern systems also are running on Windows which has interesting implications for getting into/out of places.

      That said, a lot of that stuff doesn’t touch the outside internet at all unless someone has done something horribly wrong. Medical records systems often do, though (including for billing and insurance stuff).

    • variants@possumpat.ioOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      7 months ago

      I was just watching this show called Connections and the first episode was about a power blackout and it showed how the lights went out during a birth.

      Great show it went on about what do you do if the power stays off permanently and how we aren’t well prepared for that and how to start a civilization after you kill some farmers and steal their land but non of their tools work without power either and if you know how to mount an old-school plow to oxen

  • danc4498@lemmy.world
    link
    fedilink
    English
    arrow-up
    80
    arrow-down
    1
    ·
    7 months ago

    Is there a good eli5 on what crowdstrike is, why it is so massively used, why it seems to be so heavily associated with Microsoft and what the hell happened?

    • Baggie@lemmy.zip
      link
      fedilink
      arrow-up
      101
      arrow-down
      1
      ·
      7 months ago

      Gonna try my best here:

      Crowdstrike is an anti-virus program that everyone in the corporate world uses for their windows machines. They released a update that made the program fail badly enough that windows crashes. When it crashes like this, it tries to restart in case it fixes the issue, but here it doesn’t, and computers get stuck in a loop of restarting.

      Because anti-virus programs are there to prevent bad things from happening, you can’t just automatically disable the program when it crashes. This means a lot of computers cannot start properly, which means you also cannot tell the computers to fix the problem remotely like you usually would.

      The end result is a bunch of low level techs are spending their weekends manually going to each computer individually, and swapping out the bad update file so the computer can boot. It’s a massive failure on crowdstrikes part, and a good reason you shouldn’t outsource all your IT like people have been doing.

      • themeatbridge@lemmy.world
        link
        fedilink
        arrow-up
        78
        ·
        7 months ago

        It’s also a strong indicator that companies are not doing enough to protect their own infrastructure. Production servers shouldn’t have third party software that auto-updates without going through a test environment. It’s one thing to push emergency updates if there is a timely concern or vulnerability, but routine maintenance should go through testing before being promoted to prod.

        • PainInTheAES@lemmy.world
          link
          fedilink
          arrow-up
          41
          ·
          7 months ago

          It’s because this got pushed as a virus definition update and not a client update bypassing even customer staging rules that should prevent issues like this. Makes it a little more understandable because you’d want to be protected against current threats. But, yeah should still hit testing first if possible.

          • suction@lemmy.world
            link
            fedilink
            arrow-up
            22
            ·
            7 months ago

            If a company disguises a software update as a virus definition update, that be a huge scandal and no serious company should ever work with them again…are you sure that’s what happened?

            • PainInTheAES@lemmy.world
              link
              fedilink
              arrow-up
              1
              arrow-down
              1
              ·
              7 months ago

              Ah, was a bit off. The update disregarded update controls per reddit and I must have misunderstood what exactly the channel update did. I know for the sensors you can set how closely you want to track current releases but I guess the driver update is not considered under those rules. I use CrowdStrike in my day to day but not from the administrative side, sorry for the misinformation. Thanks for the details Gestrid.

        • Baggie@lemmy.zip
          link
          fedilink
          arrow-up
          11
          ·
          7 months ago

          100% agree. I haven’t been on the backend of managing crowdstrike so I don’t know if this is a option, but running a wsuz server and manually weeding out bad updates was such an improvement over rawdogging windows updates.

      • Flying Squid@lemmy.world
        link
        fedilink
        arrow-up
        3
        arrow-down
        4
        ·
        7 months ago

        Really there’s a sub-joke here about how, because no one ever bothers scanning their Mac for viruses since they think they’re virus-proof, all the Macs are functioning as the virus farms they’ve been for quite some time.

    • Captain Aggravated@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      68
      ·
      7 months ago

      Crowdstrike is a cybersecurity company that makes security software for Windows. It apparently operates at the kernel-level, so it’s running in the critical path of the OS. So if their software crashes, it takes Windows down with it.

      This is very popular software. Many large entities including fortune 500 companies, transport authorities, hospitals etc. use this software.

      They pushed a bad update which caused their software to crash, which took Windows down with it on an extremely large number of machines worldwide.

      Hilariously bad.