Automation queue backlogged
Resolved·Major outage·3 hours 19 minutesJuly 28, 202012:073:26 PMUTC
- Update · UTCIssue is resolved
- Resolved · UTCThe issue has been resolved.
- Update · UTCWe've indentified one possible root cause which could result in processed messages being rescheduled for processing again by the automation system.
- Update · UTCThe backlog of background jobs has been processed and things should be snappy again. We're continuing to investigate the root cause of this issue.
- Identified · UTCWe've identified a situation where a number is constantly rescheduling new jobs in the background job queue causing it to grow constantly.
- Update · UTCDue to a problematic release, there are quite a few messages pending to be processed and still due for delivery. We are working through the backlog and services relying on the automation system should be responsive again shortly.
- Investigating · UTC