Turnio - Delays on inbound message processing – Incident details
All systems operational
Delays on inbound message processing
Resolved
Major outage
Started about 1 month agoLasted about 11 hours
Updates
Resolved
Resolved
This incident has been fully resolved.
We experienced delays in processing incoming messages and automated journeys due to inefficient search infrastructure operations that were slowing down our platform. Our engineering team identified and fixed the underlying performance issues throughout the day.
All services are now operating normally and message processing has returned to expected speeds.
We apologize for any delays you may have experienced with your messaging and automation today.
Identified
Identified
We're still seeing occasional delays occur and we're doing further investigation as to the underlying causes.
Monitoring
Monitoring
We have released a fix to address the database operation which was causing delays. Things are looking stable again and the system is performing as expected. We will continue to monitor the situation.
Update
Update
Recent performance improvements work has changed how updates happen to contact information. We suspect that as a side effect of this, it is highlighting an un-optimised database operation which could be causing a degradation in system performance. We are still investigating and observing.