Tuesday, September 22, 2009

Service Downtime Update and $50 Featured Listing Credit

Yesterday at approximately 3AM EST, the main web server became unresponsive to command. We had executed a large file operation on the server using a 3rd party tool that much have had a memory leak or some loop issue as the server stopped responding to all commands shortly after executing it. We then attempted to restore the server by replicating the existing server which produced more versions of the unresponsive machine. We spent too long on this effort which caused the extended outage time. It was only late last night did we get most of the core functionality back online by restoring from backups on to a new machine. We are still working through any errors that are coming in which are relatively few at this time. This episode has given us a reminder of our need to review the processes in place for recovery in an outage in our new virtual server environment. We are now taking efforts to ensure that this does not become an extended issue again. Self-recovery is our main goal with little to no downtime.

Because of the issue, we are giving all users who had a sale within the past 14 days a $50 credit in Featured Listings. Expect to see the credit applied to your account in the next week.

Thank you for bearing with us,