Wednesday, January 12, 2011

Server Downtime Today - 1/12/2011

It appears the server went down today at around 10:30AM EST. It was out until approximately 11:15AM EST. We used the Amazon EC2 console to force reboot the serve which recovered itself. We are looking into the underlying cause now. We have a backup server ready to go if we need it. All orders placed during the outage will be processed as the backlog goes out. In the meantime, you can use the Send Download function to manually send out any downloads if you like.

We have been working on our database recovery model and have a good foundation in place. We are the end of implementing the full plan. Once in place, we will be able to recover within 30 minutes from any type of outage once we make the decision the previous instance will not recover. It is an interesting process of accounting for the worst-case scenario. No one wants it to happen, but you have to be prepared when it does. I think we are crafting a best practices process now.

Thank you for your understanding,

Shannon S.