Region: Azure West Europe Cluster: prod01-eu-west-1
On September 7, 2026, some customers in Azure West Europe experienced increased query latency. Query success rates and write operations were not affected.
Three storage replicas fell behind after increased delete activity coincided with scheduled retention processing. Recovery required staged restarts and changes to compaction and retention processing. Query latency returned to normal at 15:50 UTC, and the incident was resolved at 16:16 UTC.
A sudden burst of delete requests from one organization coincided with a scheduled retention pass on three storage replicas. The combined load exhausted their compaction capacity, causing them to fall behind in processing incoming data.
The replicas did not recover on their own after the delete burst stopped. Each retention pass began immediately after the preceding pass finished, leaving no window to clear the backlog. A steady stream of delete requests from a second organization also contributed to the slow recovery.
The team contacted the organization issuing the delete burst, and the burst stopped at 10:10 UTC. The affected replicas were restarted in stages, and compaction was temporarily paused to prioritize clearing the backlog.
The retention schedule was also lengthened to give replicas recovery time between passes. Backlogs subsequently decreased across all affected replicas, and query latency returned to normal.
All times are UTC.
Time Event
09:40 - Delete burst began; storage replicas fell behind.
10:01 - Alerts triggered; incident opened.
10:04 - Cause identified; originating organization contacted; status page published.
10:10 - Delete burst stopped.
10:15 - Staged restart of affected replicas began.
11:14 - First replica fully recovered; the other two continued clearing their backlogs.
12:11 - Compaction was paused on the most-affected replica.
14:29 - Retention schedule was lengthened to allow recovery between passes.
15:29 - Backlogs were decreasing on all affected replicas; status page moved to monitoring.
15:50 - Query latency returned to normal.
16:16 - Incident resolved.
The retention scheduling change was applied during the incident response. Additional work described in the investigation includes:
The scheduling safeguards and rate limits remain under evaluation; they are not described as completed fixes; however, we continue to look for ways to provide a healthy and stable environment.