Weed out duplicate files in Windows XP

We’re going to take a look at files in XP that really are duplicated – ie more than one copy is stored on your hard disk.

First, files can have the same name but not be duplicates. If you do a search for ‘desktop.ini’ or ‘thumbs.db’ you’ll probably find hundreds, but they will all have different content. The former store folder settings and the latter contain thumbnail views of images in a folder.

Reader Alan Harrison used the Duplicate File Finder that comes with the free version of Glary Utilities and found he had over 2,600 duplicated files on his hard disk. I ran the same utility on my C: drive looking for files of the same name, size and time and found 2,445.

However, further investigation showed that 600 of these were in the System Volume Information folder. In other words they were components of different Restore Points.

Each Restore Point is a standalone entity so, if certain aspects of a system have not changed between their creation, the corresponding files will be identical.

It was then I found that the utility was somewhat flawed, as those System Restore files will not have the same time stamp.

Repeating the scan with the System Volume Information folder excluded, but with all other parameters the same showed 2,231 duplicates, rather than the expected 1,845. Time to get a second opinion, I felt.

Windows XP SP2 Support Tools are free to download. Make sure you install the full version, which will get you many tools as obscure as a left-handed monkey wrench or a metric pickaxe, but the one we want is Dupfinder.exe which you can launch from the Start Menu Run box.

This offers fewer options than Glary, and takes far longer, but found a total of 33,617 duplicates consuming 6,006,932 KB. Yes that’s around 5.7GB.

The reason is it was treating various hefty Thunderbird Inbox files – under different accounts and of different sizes as duplicates.

So I installed yet another utility named Duplicate File Finder – which found 13,679 duplicated files wasting 1.81GB.

So how do all these duplicates arise – if indeed they are duplicates?

Unruly application installations account for some – several applications install fonts, for example, in their rightful place in the Windows\Fonts folder but leave copies in the program folder. I found that Thinkfree Office had installed another set of Java Runtime files in its own program folder.

Quicktime and Flash also seem to love duplication and you will also find that there are duplicates of libraries and resources in different user profiles.

And then, of course, there’s Windows itself. You’ll find, for example, copies of DLLs in Windows\System32 duplicated in System32\DLLCache.

This is meant to happen and is an essential part of Windows File Protection. Other system files will be duplicated in Service Pack folders, Uninstall folders and elsewhere. Microsoft’s Dupfinder found 6,160 duplicated DLLs occupying 1.4GB.

So what does Hands On advise? You could spend many hours tracking down and removing duplicates and it’s almost inevitable that you will remove a wrong one and something will stop working. Windows is a mess and storage is cheap. Live with it.

Seven up
The Windows 7 Release Candidate 1 is available for public download. Having installed RC1 I have found there are a few Hands-On details you should know.

The good news is that it’s the top-of-the-range Ultimate edition and it will last until June 2010. You will, however, start getting not-so-subtle reminders on 1 March 2010 when Windows 7 will shut down every two hours.

You are advised to not install RC1 on your main PC and back up your data first. There are several useful links on the download page including ‘Installing Windows’ which tells you how to create a DVD from the downloaded 2.4GB file – I used the free Active@ ISO Burner 1.1.

It’s also worth checking out the ‘Things to know’ link. A lot of people seem to be having trouble with older Nvidia chipset display adapters, myself included. Windows 7 reported that I had a ‘standard VGA adapter’ driving a ‘non-PNP monitor’ and would only allow a painfully slow and flickery refresh rate.

I managed to force the issue by downloading and installing the XP Nvidia drivers, to get a respectable 75Hz but at the cost of a Blue Screen of Death every time I tried something graphically challenging, such as running a screensaver.