Wednesday, October 10, 2007

PayLoadz Downtime 10/8/2007

Well, this is not quite how I wanted to bring the blog back to life, but I guess I don't have much of a choice. At 3 A.M. EST today the SQL Server automatically installed an update and had rebooted itself to finish the installation like it normally does. However, this time, the MSSQL agent which runs the database server failed to initialize and the database for the PayLoadz site was never loaded properly. After fooling around with it for a little while we got everything back to where it needs to be and all is running well again.

All orders placed during the downtime should be processed normally as PayPal/Google/2Checkout begin repeating the requests for the previous orders to our order processing scripts- no orders placed during this period should be lost. They all should be processed normally.

Obviously, we try to avoid this kind of thing and yet they still happen. We will likely move to a manual installation of updates, or at least move it to a time when the main team is available so we can monitor the update accordingly.

Thank you for bearing with us during this time,


p.s.- Check back here in the next few days, I have a lot of updates to deliver!