Root Cause Analysis: Query Latency on September 11, 2026

Region: Azure West Europe Cluster: prod01-eu-west-1

‌

‌

Incident report: query latency in Azure West Europe, September 11, 2026

On the morning of September 11, some customers in Azure West Europe saw slower than usual query responses. Read latency was elevated and there were occasional query timeouts. Performance returned to normal at 08:52 UTC. We're sorry for the disruption.

What happened

On September 10, a single workload ran a sustained volume of deletes. That activity left a large backlog of delete markers on every storage partition in the region.

Delete markers are normally cleared by a scheduled delete-compaction process. Until that process ran, each hourly retention pass had to work through the full backlog. Passes that usually take about three minutes were taking 10 to 15 minutes across the cluster. While a partition was in one of these longer passes it responded more slowly, and queries that spanned it were delayed.

At 07:55 UTC on September 11, one storage partition entered a self-recovery restart. Its companion replica was in the middle of a retention pass at the time, so for a short period that partition had no healthy replica available to serve queries. This was not the root cause, but the overlap added to the impact on queries touching that partition.

Alerts on query queue depth fired at 08:14 and we opened an incident. An operator restarted the companion replica at 08:25. The impaired partition completed its recovery at 08:47. The query queue then drained without further intervention, and by 08:52 it was back to normal and the alerts had cleared.

Timeline (UTC)

Time Event

Sep 10 - Sustained high delete volume from a single workload

Sep 11, 07:55 - Storage partition restarted and entered extended recovery

08:14 - Query queue-depth alerts triggered; incident opened

08:25 - Companion replica restarted

08:47 - Impaired partition completed recovery

08:52 - Query queue returned to normal; alerts cleared

Changes we are making

There are four pieces of work under way. None of them is complete yet, and we won't consider them validated until we have confirmed the effect in production.

• Retention enforcement is moving to an off-peak window. Longer passes should then fall outside business-hours query load.
• Full compactions will run more frequently, so a delete backlog can clear between scheduled runs instead of accumulating.
• Per-partition compaction throughput is being increased.
• New alerting and telemetry will tell us whether the changes above are having the intended effect.