On 29 July 2022 at 17:01:11 to 29 July 2022 at 18:45:44, users in the Azure US-East-1 region encountered read and write unavailability due to disk management rate limits from an Azure outage, causing pods to get stuck in a ContainerCreating state when pods were being restarted.
InfluxDB Cloud is built to be elastic by resizing available storage to meet the shifting storage demands of our users. As usage of the platform grows, requests are made to the cloud provider to allocate this storage with the general expectation of disk availability from the provider.
This incident was triggered by:
Manually remove invalid Persistent Volume Claims (PVCs) and nodes that had disks that were failing to detach.