Turnio - Notice history

The issue has been resolved for the 2 clusters impacted. The underlying technical root cause was a database migration causing a table lock on one of the applications' main tables. This was further complicated by automated database cleanup processes happening on the same table a the same time. Our apologies for the downtime experienced.

The fix we've applied has not yet had it's intended effect. We're continuing to investigate.

A fix is being released to the production environments impacted.

Unfortunately we pushed a faulty release to 2 of our production clusters. This is causing downtime for https://app.turn.io. We are busy issuing a fix and the systems should be online again shortly. Our sincere apologies for this inconvenience, we will provide updates here as we address the underlying issue.

The issue has been resolved.

We've found an issue where Turn is not accepting status updates on messages. This can mean that you'll experience a delay in seeing your messages marked as read, sent, or delivered. We're busy addressing this issue.

All system upgrades have been applied.

We're performing a system upgrade in response to Google's https://cloud.google.com/kubernetes-engine/docs/security-bulletins#gcp-2020-012 security bulletin on all our Google Cloud Platform hosted systems. This will involve various restarts across the platforms. Any messages received during a restart will be handled immediately as the restarts of the various components are completed.

We've observed this for a few days and it is now resolved.

We've put some more fixes in places and are observing the system's behaviour. This issue seems to be related to automatic upgrades performed by the hosting services that app.turn.io is using. This is not impacting any of the private cloud setups.

The fix we put in place is no longer resolving this issue we are experiencing with database connections. We're investigating.

The issue is resolved. We're continuing to monitor the systems.

We've found an issue with a number of numbers unable to receive messages from WhatsApp. We've identified the issue and are busy addressing it. Any messages currently pending delivery will be processed shortly.

Content is now behaving as usual on app.turn.io

We found an issue where content is no longer editable due to a change we made in the underlying content format. This has now been reverted. Please do a hard refresh to ensure you see the changes.

Collection search indexing has been optimised to reduce load on the search backend. Search query and collection performance has been greatly improved as a result. We're continuing to work on improvements to prevent this happening in future - in particular we're changing the way All Conversations is architected so that it is much faster and doesn't rely on search

Collection search indexing has been optimised to reduce load on the search backend. Search query and collection performance has been greatly improved as a result. We're continuing to work on ways to prevent this occurring in future including changing the way All Conversations is architected so that it is much faster and doesn't rely on search.

To alleviate some of the load we'll be temporarily be turning off the total user count display for collections.

Currently, Collections rely on Search functionality to find conversations that match the Collection criteria. The All Conversations Collection, in particular, puts a high load on Search, slowing down the performance across all other Collections. We're busy working on a change to how the All Conversation Collection functions to ensure that is performs faster.

Some users are experiencing issues where Collections are loading a blank screen, or partial screen. We're exploring different reasons this may be happening.