Microsoft Outage on 25th Jan 2023 MO502273

Microsoft had its corporate earnings call yesterday and posted weaker guidance.

But guess what?

Several hours later, the tech giant was hit by a networking outage that took down Azure and other services like Teams and Outlook, affecting millions of users globally.

Early Detection of Microsoft Teams and Outlook Outage

Here’s an email our customers received informing them about errors logging into the Teams service. Our superb synthetics do this!

Below is an example of the first time the Teams sensor detected the outage in our test environment. We have configured our alarms for multiple global sites.

After 2 consecutive alarm errors within five minutes, you will see this type of email notification.

Exoprise successfully detected and confirmed Teams outage at 2:21 am last night for a particular region or user site; that’s at least  1 hour 28 mins before Microsoft diagnosed the root cause. (see below)

Microsoft Status Update on Outages MO502273

The following are a series of messages you can see in the admin center.

January 25, 2023 2:27 AM

Title: Users may be unable to access multiple Microsoft 365 services User Impact: Users may be unable to access multiple Microsoft 365 services. Current status: We’re investigating a potential issue and checking for impact to your organization. We’ll provide an update within 30 minutes.

January 25, 2023 2:51 AM

Title: Users may be unable to access multiple Microsoft 365 services User Impact: Users may be unable to access multiple Microsoft 365 services. More info: We’ve received reports that the following services are impacted: -Microsoft Teams -Exchange Online -Outlook -SharePoint Online -OneDrive for Business -Microsoft Graph Current status: We’ve identified a potential networking issue and are reviewing telemetry to determine the next troubleshooting steps. Scope of impact: Any user serviced by the affected infrastructure may be unable to access multiple Microsoft 365 services.

January 25, 2023 3:49 AM

Title: Users may be unable to access multiple Microsoft 365 services User Impact: Users may be unable to access multiple Microsoft 365 services. More info: Impact is occurring to the following services but is not limited to them: -Microsoft Teams -Exchange Online -Outlook -SharePoint Online -OneDrive for Business -Microsoft Graph -PowerBi -M365 Admin Portal -Microsoft Intune Current status: We’ve isolated the problem to a networking configuration issue, and we are analyzing the best mitigation strategy to address it without causing additional impact. We’ll provide more information once we have additional information. Scope of impact: Any user serviced by the affected infrastructure may be unable to access multiple Microsoft 365 services.

Only at this point (3:49 am), Microsoft confirms the problem to a network change.

January 25, 2023 9:28 AM

Title: Users may have been unable to access multiple Microsoft 365 services User Impact: Users may have been unable to access multiple Microsoft 365 services. More info: Impact was to the following services, but was not limited to them:  -Microsoft Teams  -Exchange Online  -Outlook  -SharePoint Online  -OneDrive for Business  -Microsoft Graph  -Power BI  -Microsoft 365 admin portal  -Microsoft Intune  -Microsoft Defender for Cloud Apps, Identity and Endpoint.  Users who could access may have experienced degraded feature functionality within the services.  Final status: We’ve confirmed, after a period of monitoring, that the majority of impacted services have been recovered and remain stable. We’re investigating some potential impact to the Exchange Online Service when connecting through Outlook on the web. For further information on the impact to the Exchange Online service please see EX502694 in the Service Health Dashboard. Scope of impact: Any user serviced by the affected infrastructure may have unable to access multiple Microsoft 365 services. Start time: Wednesday, January 25, 2023, 2:05 AM (7:05 AM UTC) End time: Wednesday, January 25, 2023, 7:43 AM (12:43 PM UTC) Preliminary root cause: A wide-area networking (WAN) routing change resulted in users being unable to access multiple Microsoft 365 services.  We’ll publish a post-incident report within five business days.

As of the latest update, our sensors are still reporting errors for various services in spite of Microsoft indicating that everything is solved.

PublishedTime:Wed, 25 Jan 2023 12:44:51 +0000MessageText:The majority of services have recovered, and the service is stable. Engineers are continuing to take actions to investigate and mitigate any residual impact caused by this incident. This quick update is designed to give the latest information on this issue.

Preliminary Root Cause for MO502273

We’ve isolated the problem to networking configuration issues, and we’re analyzing the best mitigation strategy to address these without causing additional impact. Refer to the admin center MO502273 or https://msft.it/6018eAldp for more information.

— Microsoft 365 Status (@MSFT365Status) Jan 25, 2023

You Need A Long-Term Solution for Detecting Outages

Exoprise synthetics proactively detect ALL Microsoft 365 outages so ITOps can get alerts 24*7 when a service is down or unavailable. Make your monitoring stress free and invest in a solution that actually works.

You don’t have to wait that long to confirm an outage and see who all are impacted in your office.

Plus, what happens when the change is rolled back? Does your team have visibility if the service is up and running?

Detect outages early, plus know when it’s fixed.

Contact us today and get answers to your burning questions.