Прекъсване на Smartsheet на 26 август

Sheets in Smartsheet US are failing to load

1. Решен · 4:19

   Between August 26 and August 28, 2026, customers experienced intermittent issues logging in to Smartsheet and loading sheets, Dashboards, Reports, and certain premium applications, including Control Center, Dynamic View, WorkApps, and Resource Management. The most significant disruption occurred on August 26, lasting approximately 3 hours and 56 minutes, from 12:21 to 16:17 UTC. A shorter recurrence affected customers for approximately 47 minutes on August 27, from 13:03 to 13:50 UTC. A smaller number of customers in our European region experienced a brief period of similar impact on August 28, lasting under 30 minutes, before the issue was fully and permanently resolved. Customers in our Australia and Government regions were not affected at any point during this incident. The underlying cause traced back to a routine change to an internal feature setting. That change had an unintended side effect: it exposed a software defect that caused customers' browsers to send a large, unexpected volume of diagnostic messages back to our servers. Because each of these messages required a standard identity check, they placed significant additional strain on the systems responsible for verifying that users were properly logged in. The sudden surge overwhelmed the systems and as a result, verification requests began timing out — which is what caused the login and access errors customers experienced. Our engineering teams responded quickly by adding capacity to the affected systems between 14:55 and 15:45 UTC on August 26, which restored access by 16:17 UTC that same day. However, that added capacity was not preserved in our standard configuration, so a routine software deployment at approximately 06:11 UTC on August 27 reset it, allowing the same conditions to briefly recur beginning at 13:03 UTC that day; we again resolved it by restoring capacity, with service fully stable by 13:50 UTC. On August 28, at approximately 08:50 UTC, we identified the true root cause — the feature-setting change — and deployed a permanent fix by 13:00 UTC that corrected the underlying defect, fully resolving the issue across all regions by 14:51 UTC. To prevent a recurrence, we are hardening our authentication subsystem to handle such request spikes with autoscaling, faster failure detection, request throttling and load-shedding. We are also strengthening our internal review process for feature-related changes to catch this class of issue before it reaches production. We know how disruptive this incident was, and we sincerely apologize for the impact on you and your business. We remain committed to learning from this event and continuing to invest in the reliability our customers depend on.

2. Решен · 18:23

   This incident has been resolved.

3. Наблюдаваме · 16:22

   A fix has been implemented and we are monitoring the results.

4. Идентифициран · 16:08

   We have implemented a fix to mitigate the issue causing sheets in the US Smartsheet instance to fail to load, as well as the impact on Reports, Dashboards, and Premium Apps (Dynamic View, WorkApps, Resource Management, and Control Center). System metrics are currently showing recovery, and our team is actively monitoring the service to ensure full stability. We will post further updates until the incident is fully resolved.

5. Разследва се · 15:47

   We are continuing to investigate this issue.

6. Разследва се · 15:17

   We're still investigating an issue where some sheets in the US Smartsheet instance are failing to load. This issue is also impacting Reports and Dashboards, as well as Premium Apps including Dynamic View, WorkApps, and Resource Management. Our team remains fully engaged in addressing the situation. We will continue to provide frequent updates until we reach a monitoring status.

7. Разследва се · 14:39

   We're still investigating an issue where some sheets in the US Smartsheet instance are failing to load. Our team is fully engaged in addressing the situation. We will provide frequent updates until we reach a monitoring status.

8. Разследва се · 13:28

   We're investigating an issue where some sheets in the US Smartsheet instance are failing to load. Our team is fully engaged in addressing the situation. We will provide frequent updates until we reach a monitoring status.