Tuesday, 2017-01-10

FoxyCart Service Degraded Service for Some CSS Assets

2017-01-09 3:38pm CST: One of our webservers had an issue, and stopped serving traffic. Service was not impacted. Though our notification systems did alerts of to the issue, we didn't take immediate action because there was no related service interruption.

2017-01-10 2:30am CST: We added a new webserver to replace the old, but due to an unrelated issue, this new server wasn't able to be fully automated in the build. Though it was serving traffic, it was missing a Ruby gem that was required for certain CSS generation. As a result, some requests to CSS files errored. This was mitigated by our CDN's caching (which cached the assets on subsequent loads).

2017-01-10 10:10am CST: We resolved the issue and replaced the bad webserver.