Sự cố GitHub ngày 21 tháng 10

Delays for Pull Requests and Webhook deliveries

1. Đã khắc phục · 19:22

   This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

2. Đang điều tra · 19:22

   Pull request updates, Actions, and webhook deliveries have recovered. Audit log delays are now below 40 minutes, within the healthy threshold, and continue to decrease as the remaining backlog is processed.

3. Đang điều tra · 18:52

   Audit log events continue to recover, with delays now below 70 minutes. We’ve applied mitigations to speed recovery and expect audit logs to fully recover in about 40 minutes. Pull request updates, Actions, and webhook deliveries remain recovered. We are monitoring progress and will share further updates.

4. Đang điều tra · 18:18

   Audit log events are recovering, with delays now below 90 minutes as the remaining backlog is processed. We’ve reduced load on the event-streaming platform and are monitoring recovery. Further updates will follow.

5. Đang điều tra · 17:09

   Pull request updates, Actions, and webhook deliveries have recovered. Audit log events are recovering and delays are decreasing, but we expect the remaining backlog to take approximately two to three hours to clear. We’re monitoring recovery and will share further updates.

6. Đang điều tra · 16:59

   Actions and webhook deliveries have recovered. Audit log events are still delayed by about 100 minutes. We are working through the backlog, and full recovery will take some time. We will share more updates as recovery continues.

7. Đang điều tra · 16:57

   Actions and Webhooks are operating normally.

8. Đang điều tra · 16:33

   Pull Requests is operating normally.

9. Đang điều tra · 16:25

   Audit log events are delayed by up to 100 minutes, and Actions workflow status updates continue to experience extended delays. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring recovery and will share further updates as we learn more.

10. Đang điều tra · 16:17

    Audit log events and Actions workflow status updates continue to be delayed. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring the recovery and will share further updates as we learn more.

11. Đang điều tra · 15:58

    We continue to see delays affecting pull request updates, webhook deliveries, and audit log events. The event streaming platform is showing signs of recovery, and we are monitoring as the impact continues.

12. Đang điều tra · 15:36

    We are investigating delays affecting pull request updates and webhook deliveries. Increased load on our event streaming platform is contributing to hotspots. We are shifting traffic away from affected areas and adding processing capacity; impact is ongoing while these mitigations take effect.

13. Đang điều tra · 15:15

    Actions is experiencing degraded performance. We are continuing to investigate.

14. Đang điều tra · 14:52

    We are investigating reports of degraded performance for Actions, Pull Requests and Webhooks