Збій Box 24 серпня

[Medium] Issue with Uploads

1. Вирішено · 22:37

   We recently addressed issues affecting file uploads and other write operations within Box. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 02:03 PM and 02:24 PM PDT on August 24, 2026, some users may have experienced difficulties while working in Box. During this time, users saw elevated error rates and reduced availability for uploads and other write-oriented actions. The issue occurred when a background user-cleanup process generated a sudden surge of database update activity that was amplified by an unbounded retry loop, which overloaded a database shard and exhausted connection capacity. We restored normal service by applying a rate limit to the cleanup traffic and stopping the offending cleanup workload. After traffic subsided, we removed the temporary limits once systems were stable. In addition, we are making changes to limit background cleanup traffic and improve retry behavior to reduce the chance of recurrence. # Analysis Following this incident, we are now improving how similar types of background jobs are managed and retried, and making adjustments to better prioritize customer-facing operations over these tasks. Specifically, during this issue, rapid retry cycles amplified service friction, highlighting areas where additional safeguards can further protect against similar issues occurring again in the future. We are also reviewing rollout and feature-flag practices to avoid accidental large-scale job generation. # Corrective Actions Box has initiated the following corrective actions: * Implementing bounded exponential-backoff retries for background cleanup jobs. * Enforcing throttling and monitoring on background cleanup traffic so it cannot overwhelm write-path resources. * Introducing priority-aware protections so low-priority background work is shed before it affects customer-facing operations. We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. Sincerely, The Box Team

2. Вирішено · 22:03

   After further monitoring, this incident is now considered resolved. If you continue to experience any issues, please contact Box Support at https://support.box.com.

3. Моніторинг · 21:29

   Our team has taken steps to remediate this issue and is seeing improvement for the Upload service. We are continuing to monitor for any additional impact.

4. Моніторинг · 21:29

   Our team is investigating an issue with Uploads. Users may see errors or slowness when attempting to upload items to Box. We will provide additional information as it becomes available.

5. Визначено · 21:29

   Our team has taken steps to remediate this issue and is seeing improvement for the Upload service. We are continuing to monitor for any additional impact.

6. Визначено · 21:26

   Our team has identified the underlying cause of this issue and is working to take remediating steps. We will provide additional updates as they become available.

7. Вивчається · 21:21

   Our team is investigating an issue with Uploads. Users may see errors or slowness when attempting to upload items to Box. We will provide additional information as it becomes available.