All Systems Operational
Log Ingestion (Agent/REST API/Code Libraries) ? Operational
90 days ago
100.0 % uptime
Today
Log Ingestion (Heroku) ? Operational
90 days ago
100.0 % uptime
Today
Log Ingestion (Syslog) ? Operational
90 days ago
100.0 % uptime
Today
Web App Operational
Search Operational
Alerting Operational
Livetail Operational
90 days ago
96.47 % uptime
Today
Parsing Operational
Filtering Operational
Core Services Operational
Archiving ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Past Incidents
May 9, 2021

No incidents reported today.

May 8, 2021
Resolved - All logs submitted to our service are now available when running searches.
May 8, 19:22 UTC
Investigating - Searches upon the content of logs are not returning the latest results at this time. Live Tail still shows recent logs, however. Our engineers are currently investigating.
May 8, 18:36 UTC
May 7, 2021

No incidents reported.

May 6, 2021

No incidents reported.

May 5, 2021

No incidents reported.

May 4, 2021

No incidents reported.

May 3, 2021
Postmortem - Read details
May 4, 18:20 UTC
Resolved - Alerting and Live Tail are working normally and newly submitted logs are available in the UI. Archives are currently being uploaded to their destinations. All services are fully operational.
May 3, 00:54 UTC
Monitoring - New logs are being made available in the UI at normal rates. Live Tail and Alerting have been re-enabled and returned to normal operation. We are working on processing the backlog of archives and monitoring the system.
May 2, 22:20 UTC
Update - New logs are appearing in the UI quickly. Live Tail and alerting continue to be halted. Archived logs may appear with delays for some customers.
May 2, 12:58 UTC
Update - New logs are appearing in the UI much quicker than before. Live Tail and alerting are still halted while we are backfilling logs. Archived logs may appear with delays for some customers.
May 2, 05:57 UTC
Update - New logs are now being made available quickly in our UI, with a small number of exceptions (~1%). Live Tail and alerting continue to be halted. We are working on sending archives to their proper destinations.
May 1, 19:17 UTC
Update - New logs are still appearing in the UI with delays. Live Tail and alerting continue to be halted while we are backfilling logs. Some customers may experience delays in excess of 72 hours for archived logs to appear in their archive destination.
May 1, 17:53 UTC
Update - New logs are still appearing in the UI with delays for some customers. Live Tail and alerting have been halted while we are backfilling logs.
May 1, 05:50 UTC
Update - Our investigations suggest this incident began with an update to our messaging bus / parsing pipeline. We've successfully rolled back to an earlier version and the process of backfilling logs for real-time search has begun. To speed up this process, we are continuing to keep Live Tail and alerting halted. Newly submitted logs are appearing, but still with significant delays. Log ingestion continues to work normally and no log lines have been lost.
Apr 30, 21:12 UTC
Update - Live Tail and alerting remain stopped as we solve this incident. New logs are still appearing in the UI but with a significant delay for all customers.
Apr 30, 12:38 UTC
Update - New logs are still appearing in the UI with delays for some customers. Live Tail and alerting have been stopped as we continue to investigate.
Apr 30, 05:39 UTC
Update - As we continue working on this issue, we have stopped Live Tail and alerting. New logs are still appearing in the UI, but with delays.
Apr 29, 18:42 UTC
Update - We continue to work on restoring the service to full operation. Users will still continue to experience delays in when searching views, boards and screens. Some users may not see live tail, neither receive new alerts or getting new logs.
Apr 29, 09:01 UTC
Update - As we continue working on this issue, we have temporarily scaled down live-tail and alerting. Some customers may not see live tail and receive new alerts. New logs still appear in the UI with delays.
Apr 29, 02:43 UTC
Update - We continue to work on restoring the search service to full functionality. All users will continue to see delays of greater than 30 minutes when searching, using views, boards and screens.
Apr 28, 22:31 UTC
Update - We continue to work through completing the aforementioned procedure that is still affecting a small percentage of customers. All users will continue to see delays of greater than 30 minutes when searching, using views, boards and screens.
Apr 28, 19:24 UTC
Update - We’ll shortly perform a procedure that will cause all logs to be unavailable to a small percentage of customers for a short period of time. All users will continue to see delays of greater than 30 minutes when searching, using views, boards and screens.
Apr 28, 16:36 UTC
Update - We continue to work on restoring the service to full operation. At this time users will still continue to experience delays of greater than 30 minutes when searching views, boards and screens.
Apr 28, 15:00 UTC
Identified - We have identified the issue and we are working towards restoring the service. At this time users will still continue to experience delays of greater than 30 minutes when searching views, boards and screens.
Apr 28, 13:39 UTC
Investigating - We are currently experiencing delays with search, users will experience delays of greater than 30 minutes when searching views, boards and screens.
Apr 28, 12:37 UTC
May 2, 2021
May 1, 2021
Apr 30, 2021
Apr 29, 2021
Apr 28, 2021
Apr 27, 2021

No incidents reported.

Apr 26, 2021

No incidents reported.

Apr 25, 2021

No incidents reported.