Emails Are Not Being Sent
Incident Report for Mezmo Status Page
Postmortem

Start Time: Thursday, October 28, 2021, at 16:56:52 UTC

End Time: Thursday, October 28, 2021, at 22:17:24 UTC

Duration: 5:20:32

What happened:

Email notifications of all kinds, including from alerts, were delayed for about 5 hours.  Notifications sent by Slack and Webhooks were not affected. 

Why it happened:

Our email service provider (Sparkpost) experienced an incident that caused delays for all emails from the LogDNA service.  We rely on this service to deliver email of all kinds, including notifications for alerts.  Email messages were delayed and queued until our email service provider was able to recover.  

More information on the incident can be found at Sparkpost’s Status Page: https://status.sparkpost.com/incidents/bwl8dr6gwmts?u=ydzrh5x205pf

How we fixed it:

No remedial action was possible by LogDNA. We waited until the incident from Sparkpost, our email hosting provider, was resolved.

What we are doing to prevent it from happening again:

For this type of incident, LogDNA cannot take proactive preventive measures.

Posted Nov 03, 2021 - 18:28 UTC

Resolved
Our email alerting feature has been restored to normal operation. All services are fully functional.
Posted Oct 28, 2021 - 23:22 UTC
Identified
Our email provider reports that outbound message delivery has resumed but it is not yet fully operational. Our provider will keep unsent emails in their queue and continue to try to send them.
Posted Oct 28, 2021 - 21:02 UTC
Investigating
Our email alerting feature is not working at the moment and customers are not receiving alerts by email in US region. This is due to an ongoing incident with our email provider; see https://status.sparkpost.com/incidents/bwl8dr6gwmts?u=ydzrh5x205pf for more detail. Other types of alerts, such as Slack and webhook, are still working. We are investigating.
Posted Oct 28, 2021 - 18:13 UTC
This incident affected: Log Analysis (Alerting).