Own from Salesforce - [Hipaa4] [Archive] Upstream Azure Outage Causing Service Degradation – Incident details

Archive experiencing degraded performance

[Hipaa4] [Archive] Upstream Azure Outage Causing Service Degradation

Monitoring
Degraded performance
Started 4 days ago

Affected

HIPAA4

Degraded performance from 5:04 AM to 12:00 AM

Archive

Degraded performance from 5:04 AM to 12:00 AM

Updates
  • Monitoring
    Monitoring

    The search service has been restored to full capacity. Working with Elastic support, we completed the recovery actions and the cluster is healthy. Background rebalancing across the cluster is still in progress. We are monitoring until it completes.

  • Update
    Update

    Microsoft Azure has confirmed a datacenter power outage as the source. Elastic Cloud (hosted on Azure) is degraded, which is impacting our cluster performance. Customer-facing services are not affected at this time. We are waiting on Elastic Cloud to confirm recovery

  • Identified
    Identified

    Root cause identified: an active Microsoft Azure incident is impacting upstream dependencies for our services. We are assessing scope and affected components.

  • Investigating
    Investigating

    We are investigating reports of degraded service. Upstream provider (Microsoft Azure) is reporting an active incident, and we are assessing impact on our services.