Skip to content

Comment on Microsoft Reveals How a Hidden Windows Bug Took Down Microsoft 365 Services Worldwide

You are viewing one comment and its replies. View all comments

Comments

  • Interesting. A client side bug causes server-level overload but only after unrelated changes gradually make the problem worse. Rerouting traffic to healthy servers overloads those too and spreads the outage. The client code surely “worked on my computer” lol

    40
    • Where did they say it was a client side bug?

      33
      • It says multiple times that the issue was with Microsoft’s internal cloud server software, not Windows.

        The defect came from a fix prepared specifically for Microsoft 365 networking gateway servers.

        The failure was within Microsoft’s cloud infrastructure, not customer Windows systems

        51
        • Yeah exactly, I don’t understand how the original comment is still getting upvotes.

          Since your comment it gained (+36, -11). People don’t read, I guess.

          8
      • I guess I misunderstood it.

        Inside an internal Windows process, competing tasks were waiting for access to shared resources, a problem called lock contention. Under certain traffic conditions, this exhausted processor and memory capacity.

        I read this as the behavior that precipitated the issue on the gateway was caused by how Windows sent traffic to exchange servers. So by Windows they mean whatever is running on the gateway?

        2
    • Probably.

      Joking aside, the problem with testing is that you normally cannot have a complete copy of the production env to test against, with all the possible cases.
      You may have a test env that do not trigger certain corner case scenario which can be dependent on something completely out of the domain, like in this case.

      If they learned something, they would add some way of testing even this kind of scenario in the future.

      0

Sign in to comment. Sign in