System Down

• 1:00 PM PDT (UTC-7)

  We have rolled out a hotfix that eliminates the unintended hard dependency on the third party service. We now need to reboot this service in order to have it fully recover from the earlier attack, which will result in charts and graphs being unavailable in the content management system for up to an hour. After the third party system comes back up, we will resume processing listen data, so you may see stats lag behind near real time until we catch back up on processing. Thank you for your patience.

  11:53 AM PDT (UTC-7)

  After 31 minutes of downtime, the system has recovered. This outage was caused by an unintended hard dependency on a third party service that got overloaded during the previous downtime event. We are in the process of rolling out a hotfix to remove that unintended dependency. We apologize for being down and take all outages very seriously. After we roll out the hotfix, we will work to mitigate the effect of future denial of service attacks.

  11:22 AM PDT (UTC-7)

  Our systems have alerted us that ART19.com cannot be reached. We are continuing to investigate the issue.

  10:42 AM PDT (UTC-7)

  After 16 minutes of downtime, the system has recovered. We are looking into root causes for this issue, but it appears that we have been the target of a denial of service attack. We are scaling up our systems in order to limit the effect of this attack.

  10:26 AM PDT (UTC-7)

  Our systems have alerted us that ART19.com cannot be reached. We are looking into the issue.