Between 22:00hrs on 3rd Feb 2021 and 23:59hrs on 4th Feb 2021 a software defect with bucket naming prevented data from being written correctly for a subset of buckets for a small number of customers. This defect was identified and fixed. All data that was written incorrectly has been repaired and re-written.
The AWS us-east-1 and Azure us-east-1 regions are fully operational at this time.
We are in the process of restoring any previous data for the other regions that could have been impacted during this time.
Posted Feb 05, 2021 - 01:34 UTC
Update
us-west-2 has been fully restored at this time.
Posted Feb 05, 2021 - 00:19 UTC
Update
Historical data has been restored for all buckets in us-west-2. The last hour of data will be restored in a final pass at the top of the hour.
Posted Feb 04, 2021 - 23:46 UTC
Update
Write functionality has been restored. We are additionally in the process of restoring any previous data that could have been impacted during this time.
Posted Feb 04, 2021 - 22:50 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Feb 04, 2021 - 20:58 UTC
Investigating
We are currently investigating this issue.
Posted Feb 04, 2021 - 20:57 UTC
This incident affected: AWS: Frankfurt, EU-Central-1 (Persistent Storage), AWS: Oregon, US-West-2-1 (Persistent Storage), Google Cloud: Iowa, US-Central (Persistent Storage), Azure: Amsterdam, West Europe (Persistent Storage), Azure: Virginia, East US (Persistent Storage), and AWS: Virginia, US-East-1 (Persistent Storage).