Thoughts for the Week

The need for a good and consistent backup solution for your work and personal data has very highlighted this week. We had a major system failure this past week at work and I had a system failure at home.

Work Failure

    Thursday April 12, 2007 we had a disk problem starting at about 21:30. This evidently started out slow, but everyone was at home and didn’t notice. Friday morning the system was locked up and the 2.5TB disk that contained all out file data was unreadable. Fortunately we have a solid backup system (Tivoli Storage Manager) that I personally maintain. I have no problems saying that given enough time I could recover all the data that was on the fileserver. However there’s a catch – “Given enough time” – it’s Tuesday the following week and we are still restoring data.

    We never tested restoring from scratch our data, especially certain shares with about 1TB of really small document and excel files. Restoring one really large file that’s about 2GB and then restoring 2000 – 1K files are very different even though the amount of data is about the same. The difference is in the TCP and disk overhead in the opening and closing of the files, with the large file you only do it once and with the smaller ones it’s 2000 times. We are going to propose a solution that will prevent this problem in the future, probably using the SAN to image the LUN.

Home Failure

    The same day that we noticed the failure at work I come home looking to download the latest Stargate SG-1 & Startgate Atlantis episodes from iTunes and my computer is not powering on, but it’s beeping like mad. Normally this indicates a memory problem so I pull some memory I had available to test with and the system comes up, no problem. I walk off for a minute and the system has rebooted; odd. Maybe the memory I put in had some issues, so I test and try to eliminate pieces, etc. After the weekend of trying to restore data at work and trying to troubleshoot memory at home I’m entertaining the idea that the system is dead. I finally get it to boot one last time and I get the NTLDR not found error. (If you need the explanation beyond “very bad” Google it.)     I finally remember I had the memory that originally came with the system and put it in, congratulations it booted into the OS!

Here are the issues I would have had moving to a new system:

• I have two drives that mirror each other (RAID 1) using a hardware controller – normally a good idea, however I am using the controller on the system board so if that dies my RAID array is dead and I don’t know what will happen if I connect those drives to a different controller. That means that all of my pictures I have ever taken with a digital camera could potentially be lost. The only option I would possibly have to recover the data is to send it to a data recovery service and hope it isn’t too expensive.
• I use Adobe Lightroom to manage all my photography – it’s a great program! It keeps a database of all the pictures and edits that have been made to them. I shoot in Camera RAW so I never actually edit the picture itself, so I always have the original. If I lose that database and not my pictures, I lose all my edits and any work that I have put into those pictures after the click of the shutter.
• I have a lot of data on my system that is beyond photography:
  • Automobile information
  • Financial Information
  • Personal Projects (Code & Databases)
  • Emails & Contacts

I’m sure everyone can see why this would be a total disaster, but without a solid backup solution that’s exactly what it would be. Luckily in this situation I was able to fix the problem (I still need more memory), but I know a lot of people wouldn’t be able to and if it was an actual system board failure I would have had no options. I researched a solution and I found an Iomega product that is a network server and is a RAID system holding 1TB of disk, four 250GB disks. Using RAID 5 you could get about 750GB of usable space from this. Not a bad solution if that is where you put your backup data, however knowing myself and people in general it would become primary storage and now you have no backup system anymore.

Now, what do you do if the house catches fire and destroys the disk device? Offsite storage is the only way to go here, and I’m at a loss for workable solutions. Does anyone out there have a good cost effective solution for a good offsite solution that can house a large amount of data, keep it private and secure, but if I lose the system that originally had the data, can restore it to a new system with minimal trouble.