Last saturday I was updating our Sun servers with some patches when one of them gave an error after a reboot and didn't come up anymore. There was a problem with the database that defines how the disk mirrors are configured, but it took me a while to find that out. I'll keep this short and say that I now have a lot more experience with booting from cd, repairing filesystems and software mirroring (I had to build a new database from scratch). Obviously this took way too long for one person who is learning along the way so we decided to switch the application that used a database on the faulty server over to the shadow database on our 'disaster recovery' server which is located in a different city. Monday I got the server back on it's feet, tuesday we moved the database back to the server and tuesday evening we switched the users back. Wednesday everything was running smooth again.
So, we can cross the disaster recovery test off our list for this year: it worked!
No comments:
Post a Comment