Sự cố Smartsheet ngày 27 tháng 8

Unexpected HTTP response code affecting login, sheet access, Control Center, Dynamic View and other premium apps

1. Đã khắc phục · 4:20

   Between August 26 and August 28, 2026, customers experienced intermittent issues logging in to Smartsheet and loading sheets, Dashboards, Reports, and certain premium applications, including Control Center, Dynamic View, WorkApps, and Resource Management. The most significant disruption occurred on August 26, lasting approximately 3 hours and 56 minutes, from 12:21 to 16:17 UTC. A shorter recurrence affected customers for approximately 47 minutes on August 27, from 13:03 to 13:50 UTC. A smaller number of customers in our European region experienced a brief period of similar impact on August 28, lasting under 30 minutes, before the issue was fully and permanently resolved. Customers in our Australia and Government regions were not affected at any point during this incident. The underlying cause traced back to a routine change to an internal feature setting. That change had an unintended side effect: it exposed a software defect that caused customers' browsers to send a large, unexpected volume of diagnostic messages back to our servers. Because each of these messages required a standard identity check, they placed significant additional strain on the systems responsible for verifying that users were properly logged in. The sudden surge overwhelmed the systems and as a result, verification requests began timing out — which is what caused the login and access errors customers experienced. Our engineering teams responded quickly by adding capacity to the affected systems between 14:55 and 15:45 UTC on August 26, which restored access by 16:17 UTC that same day. However, that added capacity was not preserved in our standard configuration, so a routine software deployment at approximately 06:11 UTC on August 27 reset it, allowing the same conditions to briefly recur beginning at 13:03 UTC that day; we again resolved it by restoring capacity, with service fully stable by 13:50 UTC. On August 28, at approximately 08:50 UTC, we identified the true root cause — the feature-setting change — and deployed a permanent fix by 13:00 UTC that corrected the underlying defect, fully resolving the issue across all regions by 14:51 UTC. To prevent a recurrence, we are hardening our authentication subsystem to handle such request spikes with autoscaling, faster failure detection, request throttling and load-shedding. We are also strengthening our internal review process for feature-related changes to catch this class of issue before it reaches production. We know how disruptive this incident was, and we sincerely apologize for the impact on you and your business. We remain committed to learning from this event and continuing to invest in the reliability our customers depend on.

2. Đã khắc phục · 15:53

   This incident has been resolved.

3. Đang theo dõi · 13:52

   A fix has been implemented and we are monitoring the results.

4. Đang điều tra · 13:33

   We are investigating an outage impacting login, sheet access, Control Center, and Dynamic View. Our team is fully engaged in addressing the situation. We will provide frequent updates until we reach a monitoring status.