Previous incidents

We have implemented a mitigation to enable jobs processing. Scheduled jobs are now executing as expected, and any delays to asynchronously processed tasks should clear shortly. Conversations across all channels were unaffected throughout the incident.

The service disruption affecting chat and voice channels has been resolved — all channels are now operating normally. We identified the issue, deployed a mitigation, and confirmed that latency and error rates returned to normal levels as of 11:05am PDT.

We apologize for the disruption, and for impacted customers, we will follow up with a root cause analysis (RCA)

Starting at approximately 8:15am PT this morning, a platform update caused the chat widget to render in a manner that may have obscured its content for a subset of customers. We have a fix in place as of 8:57am PT and are verifying recovery across affected customers. Widgets are rendering normally again, and we're continuing to monitor to confirm full resolution.

We have identified the root cause and pushed out a fix. We are seeing full recovery

The issue is resolved, all dashboard views and reports have complete data for May 7 and 8th.

The fix has been implemented and the issue is now resolved.

From 1:16-1:19pm PST, we had a brief period of instability where roughly 10% of conversations in the US experienced degraded flows. These were primarily voice conversations, with a small number of chat conversations also affected. We have resolved the issue and are fully operational.

We've seen no repeat errors since 9:54am PT and are marking this incident as fully resolved. We will follow up with impacted customers to share a postmortem.

This issue has now been resolved.

Impact was limited between 11:00am - 11:36am PT, during which a small number of ongoing voice calls experienced degraded performance due to higher than normal latency.

If you have any questions, please follow up with your Decagon point of contact.