Cloud provider incident
Incident Report for Catalytic
Resolved
Our cloud provider has fully recovered and we are not seeing any remaining issues.

We have seen some inbound emails that were drop be recovered, but not all. We are still waiting for a final confirmation from our cloud provider but do not believe any emails that have not yet been delivered will be. We will update here if we hear differently from our cloud provider.
Posted Dec 08, 2021 - 11:40 CST
Monitoring
Our cloud provider has resolved the issue and is almost fully recovered. There should be no issues for workflows started now, however there may be issues for workflows starting during the outage.

We have seen some inbound emails sent during the outage come through, but not all. We are still waiting on final confirmation from our cloud provider on if we should expect all emails to eventually come through.

We are now monitoring but will not close this incident until we have received confirmation from our cloud provider the issue is fully resolved and have an answer on inbound emails.
Posted Dec 07, 2021 - 20:02 CST
Update
Our cloud provider is experiencing an issue with several networking devices. They have applied mitigations that are showing significant improvement in their internal metrics that show they are on the path to recovery, but do not have an ETA for full recovery.

We've confirmed this is impacting Catalytic in several ways:

- preventing email triggers from starting workflows
- pages occasionally failing to load. We are seeing refreshing the page work after a few minutes
- trouble adding showcase templates

This incident is fluid and we are seeing the impact to Catalytic change over time.

We are waiting to hear back from our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. We are not seeing any other issue that can not be recovered after the cloud provider incident is resolved.

We are continuing to monitor if any other parts of Catalytic are impacted and communicate here when we do.
Posted Dec 07, 2021 - 17:01 CST
Update
Our cloud provider is experiencing an issue with several networking devices. They report some early signs of recovery, but do not have an ETA for full recovery.

We've confirmed this is impacting Catalytic in several ways:

- preventing email triggers from starting workflows
- pages occasionally failing to load. We are seeing refreshing the page work after a few minutes
- trouble adding showcase templates

This incident is fluid and we are seeing the impact to Catalytic change over time.

We are waiting to hear back from our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. We are not seeing any other issue that can not be recovered after the cloud provider incident is resolved.

We are continuing to monitor if any other parts of Catalytic are impacted and communicate here when we do.
Posted Dec 07, 2021 - 15:16 CST
Update
Our cloud provider is experiencing an issue with several networking devices. They report some early

We've confirmed this is impacting Catalytic in several ways:

- preventing email triggers from starting workflows
- pages occasionally failing to load. We are seeing refreshing the page work after a few minutes
- trouble adding showcase templates

This incident is fluid and we are seeing additional issues pop up over time.

We are waiting to hear back from our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. We are not seeing any other issue that will not recover when the incident resolves.

We are continuing to monitor if any other parts of Catalytic are impacted and communicate here when we do.
Posted Dec 07, 2021 - 15:05 CST
Update
Our cloud provider is experiencing an incident and is not able to provide an ETA for resolution at this time.

We've confirmed this is impacting Catalytic in several ways:

- preventing email triggers from starting workflows
- pages occasionally failing to load. We are seeing refreshing the page work after a few minutes
- trouble adding showcase templates

This incident is fluid and we are seeing additional issues pop up over time.

We are waiting to hear back from our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. We are not seeing any other issue that will not recover when the incident resolves.

We are continuing to monitor if any other parts of Catalytic are impacted and communicate here when we do.
Posted Dec 07, 2021 - 14:12 CST
Update
Our cloud provider is experiencing an incident. We've confirmed this is impacting Catalytic in several ways:

- preventing email triggers from starting workflows
- pages occasionally failing to load. We are seeing refreshing the page work after a few minutes
- trouble adding showcase templates

This incident is fluid and we are seeing additional issues pop up over time.

We are reaching out to our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. All other issues appear to resolve on their own.

We are continuing to monitor if any other parts of Catalytic are impacted and communicate here when we do.
Posted Dec 07, 2021 - 12:46 CST
Identified
Our cloud provider is experiencing an incident. We've identified this is preventing email triggers from starting workflows. We are reaching out to our cloud provider to determine if emails received during their incident will be delivered when their incident is resolved. We are continuing to monitor if any other parts of Catalytic are impacted.
Posted Dec 07, 2021 - 11:12 CST
Monitoring
Our cloud provider is experiencing an incident that is disrupting other services. We have not seen any disruptions to our service but are closely monitoring and working with our cloud provider to resolve. We will update this issue if we identify any Catalytic issues related to the cloud provider incident.
Posted Dec 07, 2021 - 10:40 CST
This incident affected: Catalytic Application and Workflow Actions.