Signal

Preliminary report: Microsoft attributes a major 365 and Exchange outage to a software defect introduced in a July 2026 update for network gateway servers

First reported by The Register ·

The signal ●●●● Compiled by AI from The Register, Techmeme, PCMag, XDA Developers, The Stack and 2 more
Why you might care

Your Microsoft 365 services, including email, experienced a multi-hour outage due to a software flaw deployed months prior.

What happened

Microsoft has attributed a major outage affecting Microsoft 365 and Exchange Online to a software defect introduced in a July 2026 update for its network gateway servers. A preliminary report indicates the defect caused lock contention, exhausting server resources, particularly on newer hardware. The outage was triggered when a Teams failover increased load on a primary gateway, coinciding with approximately 14 percent of server capacity being temporarily unavailable. This combination overwhelmed the gateway, leading to a cascading failure across Microsoft's infrastructure, rendering over 80 percent of servers unresponsive. Standard recovery processes failed, and a simple rollback was impossible due to the defect's age. Engineers developed an emergency patch to revert the code path to a stable state, gradually restoring services over several hours. The incident caused widespread disruption for over 15 hours.

What it means

The incident reveals significant shortcomings in Microsoft's load testing methodologies, which disproportionately favored older, less powerful hardware. This oversight allowed a critical defect to persist undetected, impacting newer, high-core-count servers under specific, albeit foreseeable, operational conditions. The failure of automated recovery systems further highlights a gap in their resilience strategy, forcing engineers to create a patch for an existing patch rather than a straightforward rollback.

This deep-seated defect, lurking for months, underscores the risks inherent in complex cloud ecosystems and the potential for subtle software changes to have widespread, cascading effects. The extended downtime, particularly for services like Exchange Online that cannot easily fail over, demonstrates the critical dependencies users have on Microsoft's infrastructure, and the potential consequences when those dependencies are compromised.

AI-written summary. May contain errors.