Listening Platform Distribution Delays

• 12:47 PM PDT (UTC-7) June 3rd 2022

  What happened?
  On 2022-05-31 at 3:49 PM PT a stuck job in our RSS feed invalidation service caused RSS feeds to not be updated correctly.

  What was the impact?
  Any episodes published after this time were unavailable on listening platforms dependent on RSS based distribution.

  How did we resolve the issue?
  At 7:42 AM PT we increased the amount of workers in our RSS feed invalidation service and replaced the stuck invalidation job. At 7:50 AM PT we observed new episodes becoming available on listening platforms.

  What steps are we taking to prevent future incidents?
  We have permanently increased the desired tasks in our RSS feed invalidation service. Additionally, we are adding latency based alarms to this service and are investigating further monitoring, alarming and health check mechanisms to mitigate any future issues.

  Again, we apologize for any inconvenience caused. Feel free to reach out to support@art19.com with any other questions or concerns.

  07:42 AM PDT (UTC-7) June 2nd 2022

  We are seeing recovery from the distribution delay and new episodes should be available as soon as listening platform databases are updated. We will continue to closely monitor the situation and will publish a postmortem within the next few days. We’re sincerely sorry for the inconvenience and appreciate your patience. If you have any questions or concerns don’t hesitate to reach out to us at support@art19.com.

  06:12 AM PDT (UTC-7) June 2nd 2022

  We’re are aware of reports of new episodes not appearing on listening platforms. We’re actively investigating the issue. As soon as we know more we will send a follow up. We’re sincerely sorry for the inconvenience and appreciate your patience. If you have any questions or concerns don’t hesitate to reach out to us at support@art19.com.