I had an unfortunate set of events happen in the last few weeks. The Windows XP system that I use at home was getting pretty flaky. I had hooked up my sister’s hard drive to backup / recover a few files for her and when I ran a virus checker I was a little shocked. The HDD had about 275 distinct viruses! I didn’t think that was possible. So I cleaned everything up (including spyware) and thought all was well. Unfortunately my own system that I used for the recovery started acting up. I thought that maybe one of the viruses (virii?) got onto my system somehow and was messing things up. I now know that it was just a bad coincidence. The hard drive was in the final process of dying a slow and painful (to me) death.

Anyway, long story short (maybe not short enough), I had lots of files that needed to be recovered. I had backed up all of the critical files like documents and budgets and such but I never backed up the photos from my digital camera or the approximately 35 GB of music. I don’t have any real way to back up 35 GB of music anyway. I have access to the original sources on CD or from archive.org for live concerts, but that is such a hassle to re-rip the CDs.

I started to copy files to the new hard drive but got lots of CRC errors. This was going to be painful. I needed a utility to copy the files and ignore the CRC errors. Of course I know that the section that has the CRC error is toast, gone, kaput. But that’s OK I suppose. It is better than manually copying every single file individually.

I came across Copy It Anyway! which will copy a file or set of files with read errors. Usually the whole file is not bad, just enough for one sector. With MP3 or videos this will usually be OK.

There are a few problems with this utility. First and foremost, it can’t copy subdirectories. There are a few other minor quirks I’ll explain at the end but the inability to copy whole subdirectories made this a very time consuming problem to recover my files.

I needed a batch file to loop through the old hard drive and build the command line to call CIA (Copy It Anyway!).

I have a controller batch file that coordinates the rest of things.

doit.bat:

Let’s look at what this is doing line by line.

Hard coded locaton for my source files (f:\My Music). The power of this line is the /b which tells the dir command to only list bare filenames. When used in conjunction with the /s it will also display the full path. The “/a:d” tells dir to only display directories.

This is an old DOS trick to concatenate two files. The listing below this has the contents of fragment.txt. Note that there is no line feed after the = or this won’t work.
the end part “> nul” just suppresses any error messages.

Now we want only the first line of the file. “find” will give us this. Unfortunately there isn’t a clean way to do this without a lot of temporary files.

Here we are echoing the literal string “call process.bat” to the end of temp.bat. We’re not executing it yet.

Execute temp.bat. Remember that when calling batch files from another batch file you must use “call”. Otherwise, the calling script will end as soon as the called script is finished.

Now take the temp.txt and list all files that do NOT have “set filename=” in them. We’re writing it back out to filelist.txt. So each time through the loop filename.txt will be one line smaller.

These two lines go together. Basically, if we type filelist.txt and there is still something left in the file then goto START and loop the whole thing again. If the file is empty then it will stop.

fragment.txt:

process.bat:

Let’s look at what process.bat is doing.

%filename% was populated when we ran temp.bat in the calling batch file.

Sometimes there is no real substitute for a good unix utility. Sed is a classic utility to edit the incoming stream (sed=Stream EDitor). The regular expression “s/f:/c:/g” says search for all “f:” and globally (g at the end) replace it with “c:”. If your source or destination are different then you’ll need to modify this.

Just like in the previous file, prepend a SET onto the first line of the file.

This will set the environment variable %outname% for later use.

Make sure that the destination directory is created. Technically this isn’t required, but if I tried to copy files to a directory that didn’t exist CIA.exe would prompt me to create it. I put this here so I wouldn’t have to babysit the script.

Output what’s happening.

This will run CIA with the source and destination.

fragment2.txt:

There are a few problems with this script:

• If there are no files in the destination directory CIA would display a popup that I had to click on. This meant that I did have to babysit it a little bit.
• The script didn’t end correctly. When there were no files left in the filelist.txt it went into an endless loop. That was no big deal. Ctrl-C fixed that.
• If I needed to restart the script, CIA would prompt me to overwrite existing files. So I had to manually click “No to all”. This was a bit tedious but on the whole not too bad.
• If the last file in a directory had errors then CIA would display a summary of files copied. This had to be closed manually
• The whole process was damn slow! It took almost 4 days to copy 35 GB of data. A lot of time was because I had so many CRC Read errors on the drive. It was also because of unfortunate timing on prompts from CIA.

On the whole it was an interesting experience getting all the data off the old drive. I think I recovered the vast majority of the files. Some of the MP3s are trashed because the first few bytes were corrupt. Oh well, that’s better than ripping everything again.