Below we have tested and listed the top 16 best duplicate file finder tools to remove duplicate and similar files. This comprehensive list contains the finest duplicate cleaners both free and paid. Choosing the best clone remover is the most important part of the deduplication process.
Duplicate files may easily build up in many ways – by downloading multiple times, copying, saving in different locations, backing up, and many more. They not only occupy space, but they also reduce the performance of your Windows system. A larger quantity of repeating files means more useless data to be processed by antivirus and all other types of applications leading to slower system performance.
Certainly, you don’t want to scan your valuable data with a poor tool that may produce incorrect results and lead to data loss. We will cover the key features that you should consider when choosing a duplicate remover.
Best Duplicate File Finder & Cleaner Tools to Delete Repeating Files
All software in the best duplicate finder list below is sorted from BEST to WORST.
- Fast Duplicate File Finder
- Visual Similarity Duplicate Image Finder
- Audio Dedupe
- Duplicate Photo Finder
- Duplicate Music Finder
- Folder Size
- Duplicate Finder
- Czkawka Duplicate Finder
- Dupe Guru
- Duplicate File Finder Plus
- AntiDupl
- CloneSpy
- Deduplicator
- Wise Duplicate Finder
- SearchMyFiles
- SourceForge Duplicate Files Finder
1. Fast Duplicate File Finder
Rating: OUTSTANDING
Duplicate Finder
Fast Duplicate File Finder won our BEST DUPLICATE FILE FINDER award. It is the fastest and the most feature-rich application of all that we have tested. The best part is that it has both FREE and commercial versions. Some features are not available in the free version but it is perfectly sufficient to perform a duplicate cleanup. Nevertheless, if you want to use the advanced features for finding similar files or filtering and exporting, you can always purchase a license. The Professional version is just $39.95 and you get a lifetime license that does not expire like the licenses of the other commercial tools. Read below why we have chosen this tool as the finest duplicate remover in our list.
Duplicate File Finder
The first and most important thing is the fact that this tool provides 100% accurate results. It was the only tool in our tests that never produced incorrect results. That is because it does not rely on simple CRC or MD5 checksums. It uses an optimized byte-to-byte comparison method which is both fast and reliable.
Duplicate Search Methods
The tool provides multiple scan methods. Those include the exact duplicate search mode, similar files mode, similar file names mode, and file size – equal file size mode. The exact duplicate and equal file size modes don’t need any clarifications.
The similar files mode is something that you will not find in other applications. Computers are precise digital machines and finding similarities is not something that they can do out of the box. In order to find similarities, a tool should implement advanced analysis algorithms. The developers of Fast Duplicate File Finder have implemented really comprehensive similarity search algorithms as this tool can find even text documents that have rearranged paragraphs and altered sentences. None of the so-called fuzzy search tools that we tried could detect such files as similar.
The mode for searching similar file names is extremely handy too. For example, if you have saved the same document with a prefix/suffix or a different date in the name, this feature will help you to find all its copies. Also, files that you have copied and Windows Explorer has automatically added a “copy x of” will be identified. Of course, the feature can find files with duplicate names too. Just set the similarity level to 100% and only exact duplicate file names will be listed.
Duplicate File Preview
The preview in FDFF is superior to any other tool that we have tested. It can preview ALL popular image formats (about 40) plus 300 RAW Photo image formats. It can also preview all audio and video files. Anything that you can play in Windows Media Player will be played in the application too. You can add support for extra audio and video files by simply installing codec files for them and they will be available in the application. The preview for PDF and EXCEL files is extremely handy. It makes comparing documents very easy. You do not need to double-click the files in order to open and view them in Word or Excel.
Other Notable Duplicate Finder Feature
The tool comes pre-configured with protection for system files and folders. A setting in the application options comes enabled by default. If you feel brave you can always disable it which I really do not recommend. As an additional precaution, I strongly advise you not to scan any system folders for duplicates. These include the Documents and Settings, the Windows folder, the Program Files folder, and others. As I mentioned earlier some files in those folders are intentionally duplicated and deleting them may lead to issues with the system. The best practice is to scan only your own document folders for duplicates rather than scanning entire computers or drives.
Duplicate Finder Options
The tool provides all the filtering features that you will ever need. You can limit the scan to only certain file types, sizes, names folders, dates, and any other file and folder property.
There is a feature to compare two or more folders for duplicates. It uses a clever method to exclude a folder from self-scan and auto-check. This means that the files inside the folder will not be compared with each other – they will be compared only with files residing in other folders.
Filter Duplicate Files
Fast Duplicate File Finder Verdict
This tool has both free and paid Professional versions. It is the best tool in our test and even the free version is sufficient to get the work done. If you like the tool you can support the developers with a one-time purchase of $39.95 and you will receive a lifetime license which unlike other popular tools does not expire.
2. Visual Similarity Duplicate Image Finder
Rating: OUTSTANDING
Visual Similarity Duplicate Image Finder
This duplicate finder is the second-best on our list. I should mention though that it is not a general-purpose duplicate remover. It works only on images and can find similar and duplicate photos. The reason to place it second is its unique capability to find similar photos. As I already mentioned finding similar files is not something that common clone deleting tools can do. The previous tool that I reviewed can find similar files, but it is efficient for documents, archives, and text files. You can read in this article which duplicates file finder to choose for finding documents, photos, or audio.
Duplicate Photo Finder
Duplicate Image Finder
DOWNLOAD NOW
Compatible with Windows 11/10/8.1/8/7/Vista/XP (Both 32 & 64 Bit)
VSDIF is the top application for finding similar and duplicate photos. It can find repeating pictures even if they are stored in different file formats. Edited, rotated, flipped, and converted photos are no issue and will be detected too. This tool is at the top of The Best Duplicate Photo Finder list. It is used by large corporations, medical research laboratories, and professional photographers.
No other duplicate photo finder tool provides this many features and this many supported image formats. Please note that if the format in which your photos are stored is not supported by a similarity search application it will not be able to detect them.
Duplicate Photo Finder Supported Image Formats
- JPEG Bitmap (*.jpg;*.jpeg;*.jpe)
- Compuserve Bitmap (*.gif)
- Portable Network Graphics (*.png)
- TIFF Bitmap (*.tif;*.tiff;*.fax;*.g3n;*.g3f)
- JPEG2000 Files (*.jp2)
- JPEG2000 Code Stream (*.j2k;*.jpc;*.j2c)
- Targa (*.tga;*.targa;*.vda;*.icb;*.vst;*.pix)
- Paintbrush (*.pcx)
- Windows Bitmap (*.bmp;*.dib;*.rle)
- Windows Metafile (*.wmf)
- Enhanced Windows Metafile (*.emf)
- Windows Icon (*.ico)
- Windows Cursor (*.cur)
- Wireless Bitmap (*.wbmp)
- Portable Pixmap (*.pxm;*.ppm)
- Adobe Photoshop (*.psd)
- Camera RAW (*.crw; *.cr2; *.cr3; *.fff; *.eip; *.dcs; *.drf; *.ptx; *.pxn; *.mdc; *.obm; *.nef; *.raw; *.pef; *.raf; *.x3f; *.bay; *.orf; *.srf; *.mrw; *.dcr; *.sr2; *.dng; *.erf; *.mef; *.arw) [ List of all 300+ Camera RAW formats ] (only in Pro and Corporate versions)
- DICOM Images (*.dcm) (only in the Corporate version)
- HDPhoto Images (*.hdp; *.wdp; *.jxr) (only in Pro and Corporate versions)
- WebP Images (*.webp) (only in Pro and Corporate versions)
- HEIC Images (*.heic) (requires external WIC codec) (only in Pro and Corporate versions)
Similar Images Search
We have covered this tool in detail in our Lightroom Duplicate Finder article too. If you are a Lightroom fan then you are lucky as this tool can scan your lightroom photos too. There is a dedicated “Scan Lightroom” button on the toolbar that will let you select a lightroom catalog. All the features of the application are available for finding similar and duplicate photos in Lightroom too.
Free download of VS Duplicate Image Finder.
3. Audio Dedupe
Rating: OUTSTANDING
Duplicate Songs Finder
This is the best tool for finding duplicate songs and similar audio files. It is specialized in organizing audio libraries by identifying repeating tracks and songs. Audio Dedupe made it up to third place on our list as it will “Listen” to your songs in order to find similarities. Uses advanced audio analysis algorithms and can find similar songs even if they are stored in different audio formats. It can find remixes of the same song and even the same song performed by another singer.
Duplicate Songs Finder Supported Audio Formats
This is a truly amazing application and if you want to organize your songs you can’t go wrong with it. Support all the popular audio formats:
- AAC Files (*.aac; *.mp4, *.m4a) – iTunes, iPod, iPhone Audio Files including Apple Lossless audio
- AC3 Files (*.ac3)
- Audio IFF Files (*.aif; *.aiff)
- FLAC Files (*.flac)
- MPEG Audio Files (*.mp3;*.mp2;*.mp1;*.mpa)
- Musepack Files (*.mpc)
- OGG Files (*.ogg)
- WAVE Files (*.wav)
- WavPack Files (*.wv)
- Windows Media Audio Files (*.wma)
- Monkey’s Audio Files (APE)
- True Audio Files (TTA)
Duplicate Songs Finder
Provides all the features that you will ever need in order to organize your audio library. It can also find similar ID3 audio tags and exact duplicates.
The built-in preview makes the process of reviewing the duplicates a breeze. It plays a track continuously from the same position while you navigate between the cloned songs. This way you can easily compare the quality of the files.
Of course, the tool can mark the files with the lower quality automatically in order to remove them easily. The bit rate and other information regarding the audio files is displayed in the duplicates list too.
Download Audio Dedupe duplicate songs finder.
4. Duplicate Photo Finder
Rating: OUTSTANDING
Duplicate Photo Finder is another great dedicated duplicate finder for finding similar images. It can find duplicate images as well as similar photos in any image format. As I already noted, if you are looking to organize your images a dedicated duplicate photo finder is the best choice.
Detect rotated, flipped, resized, or edited pictures. Duplicates will be identified regardless of the modifications that are applied – transformations, color corrections, crops, borders, resized pictures, and added watermarks.
Duplicate Photo Finder
Supports over 340 photo formats, including RAW, HEIC/HEIF, JPG, PNG, GIF, CRW, CR2, CR3, DNG, PSD, WebP, and many more.
The developers claim that there is no limit to the count of the scanned images. Folders, disks, removable HDD and SSD flash drives, or network shares all can be scanned for identical photos.
Find Similar Photos & Similar EXIF
The tool supports multiple scan modes. It can find similar photos as well as identical images stored in different formats. It can search by file size and EXIF date. and can also find similar EXIF fields. For example, you can search for files with a similar EXIF description. Certainly, you can also find duplicate EXIF fields. If you search for example, for a duplicate CAMERA MODEL EXIF field the tool will list all the photos shot with a particular camera together.
Find Adobe Lightroom Duplicates
In addition to the above, the tool can also find duplicate photos in Adobe Lightroom. This tool is way more advanced than any Adobe Lightroom duplicate finder plugin. It will let you choose a lightroom catalog and find all similar and duplicate photos in it. The common Lightroom duplicate finder plugins can not find similar images. They scan file attributes and in the best case some EXIF flags. In contrast, Duplicate Photo Finder can find both similar photos and similar EXIF info.
Duplicate Photo Finder Verdict
If you are looking to organize your images and get rid of duplicate pictures stored in different formats then you can’t go wrong with this tool. It goes way beyond the capabilities of the common clone removers and will find even edited, retouched, and watermarked photos. Those are features that are not available in any standard duplicate cleaner.
Download Duplicate Photo Finder
5. Duplicate Music Finder
Rating: OUTSTANDING
Duplicate Music Finder is another great entry in our list of the best duplicate file finders. It is a specialized duplicate remover tool that will find copied songs or similar audio files in your music collection. Duplicate songs are difficult to locate and remove manually. in addition to that, audio files are quite large and take up a lot of disk space. Therefore, keeping an organizing audio collection is a must. Having multiple copies of the same song in your playlist is quite annoying. It is equally annoying to have missing files and you might want to read our remove missing songs from iTunes article too.
Duplicate Music Finder Screenshot
Duplicate Music Finder
Things get worse if the songs are stored in different audio formats and not marked with ID3 tags as no common duplicate finder can recognize them. Duplicate Music Finder solves this problem by analyzing the audio tracks and recognizing similar songs. It will “listen” to your audio collection in order to identify twin songs or variations of the same audio file. It can search through various file types including MP3, MP2, MP1, MPA, WAV, OGG, AIFF, AAC, MP4, FLAC, AC3, WavPack (WV), Musepack (MPC), and Windows Media Audio (WMA), iTunes, iPods, etc.
Download Duplicate Music Finder
6. Folder Size
Rating: OUTSTANDING
Folder Size is not exactly a duplicate finder, but if you are trying to organize your disk and free up space, then it will probably be even more useful for you than a duplicate cleaner. The best part of all is that this great tool is free. It has a commercial version, but the features that you will need to clean up your drives are available in the free version.
As you can guess from the name, this tool will show all folder sizes on your computer. Folder sizes are not listed in Windows File Explorer and it is hard to spot the missing disk space. If you wonder why then read this article: Folder Sizes in Windows Explorer
Folder Size Explorer
The tool will not only list the folder and file sizes, but it will also list the largest files and largest folders. It sorts the files and folders from largest to smallest and displays the disk space distribution using neat pie and bar charts.
Folder Size Features
- View all folder sizes in Windows
- View network folder sizes, external HDD, NAS, SAN, and USB drives
- Find the largest folders and largest files
- Print folders
- Print files list
- Export folders to EXCEL
- Export files to EXCEL, XML, or CSV
- Find empty folders
- Find longest paths
- Find the newest or oldest files and folders
- Visualize disk space with neat pie and bar charts
- Save folder and files lists to project files
- Cut, Copy, and Paste files and folders from and to Windows Explorer
Folder Size Verdict
This is a fantastic little tool that is really lightweight and does not use many resources. These are just a few of the major features of the tool. You can free up gigabytes of disk space using only the free version. If you have decided to organize your files then you should definitely try it. This is a tool that I personally use daily. I am sure that you will love it too.
Download Folder Size
7. Duplicate Finder
Rating: OUTSTANDING
Duplicate Finder is another free and robust duplicate cleaner application in our list of the top cloned file cleaners. It covers the most important points of our checklist for choosing clone removal apps.
Duplicate Finder
Duplicate Finder Overview
The tool uses an optimized byte-to-byte comparison algorithm that provides accurate results. The user interface is simple and intuitive but also includes advanced features for more complex deduplication tasks. Windows 11 is not listed in the supported systems, but our test confirms that the tool is working flawlessly on that Windows version.
The free version is good enough to clean up duplicates, but the similarity modes are available only in the commercial version. Nevertheless, the options for cleaning exact copies are fully functional and are sufficient for most deduplication tasks.
Duplicate Finder Search Modes & User Interface
As I already mentioned, the byte-to-byte and file size modes are available for free. The similarity search modes can find similar file names or similar documents. You can use them in the free version too, but they display only the first five groups of similar files. It is a kind of a teaser, but if you have fewer files you can still use the features.
Scanning for duplicates is simple and easy. Drag and drop the folders that you want to scan in the app, and hit the scan button. The tool will find exact copies by default. As soon as the search is finished the cloned files are listed in the main window. The repeating items are grouped together and details for every file are listed in a neat table.
You can easily review the results and also view every file in the preview panel below the duplicates list. Dupe Finder can preview images, music, video, PDF, Word, Excel, and many other file formats. Do not underestimate the preview capabilities of a tool. Having an instant preview saves a tremendous amount of time when reviewing the clones. You see the content of the files instantly rather than opening them with the application that supports that file format.
Duplicate Finder Verdict
As with most tools you can delete, move, or backup the files. In addition to that, you can replace the duplicates with sym-links, Those tiny files are just a few bytes large and point to the “original” file. The application creates those sym-links in the place of the duplicates. Therefore if an application requires the duplicated file that was removed it will still work as if nothing was changed.
Overall Duplicate Finder is a feature-rich clone cleaner that will complete even the most specific task that requires removing repeating data.
Download Duplicate Finder
8. Czkawka Duplicate File Finder
Rating: Average
Yes, this name looks horrible to pronounce because it is written in Polish. Czkawka (tch•kav•ka (IPA: [ʈ͡ʂkafka]), “hiccup” in Polish) is a simple, fast, and free app to remove unnecessary files from your computer. This is another multi-platform duplicate finder application.
Supported OS
Linux – Ubuntu 22.04+, Fedora 36+, Alpine Linux 3.16+, Debian 12+ and a lot of more
Windows – 7, 8.1, 10, 11
MacOS – 10.15+
Czkafka Duplicate File Finder
As I have already noted, trying to support multiple operating systems simultaneously leads to compromises and issues.
Here are the features as described by the developer.
Features
- Written in memory-safe Rust
- Amazingly fast – due to using more or less advanced algorithms and multithreading
- Free, Open Source without ads
- Multiplatform – works on Linux, Windows, macOS, FreeBSD, and many more
- Cache support – second and further scans should be much faster than the first one
- CLI frontend – for easy automation
- GUI frontend – uses GTK 4 framework and looks similar to FSlint
- No spying – Czkawka does not have access to the Internet, nor does it collect any user information or statistics
- Multilingual – support multiple languages like Polish, English, and Italian
- Duplicates – Finds duplicates based on the file name, size, or hash
- Empty Folders – Finds empty folders with the help of an advanced algorithm
- Big Files – Finds the provided number of the biggest files in a location
- Empty Files – Looks for empty files across the drive
- Temporary Files – Finds temporary files
- Similar Images – Finds images that are not exactly the same (different resolution, watermarks)
- Similar Videos – Look for visually similar videos
- Same Music – Searches for music with the same artist, album, etc.
- Invalid Symbolic Links – Shows symbolic links which point to non-existent files/directories
- Broken Files – Finds files that are invalid or corrupted
- Bad Extensions – Lists files whose content does not match their extension
Of course, the tool can not compete with any dedicated duplicate image finder or duplicate music finder, but at least it is free. Follow the link below if you want to give it a try.
Download Czkawka Duplicate File Finder: https://github.com/qarmin/czkawka
9. Dupe Guru
Rating: Average
dupeGuru is a cross-platform (Linux, OS X, Windows) GUI tool to find duplicate files in a system. I usually do not recommend using cross-platform tools. That is because creating a cross-platform tool leads to a lot of compromises. It is like creating a car that can fly and be a submarine at the same time. You can understand that in the end, it will not do any of the three tasks well.
It’s written mostly in Python 3 and uses multiple GUI toolkits, all using the same core Python code. On OS X, the UI layer is written in Objective-C and Cocoa. On Linux & Windows, it’s written in Python and uses Qt5.
dupeGuru is a tool to find duplicate files on your computer. It can scan either filenames or contents. The fuzzy matching algorithm can find duplicate filenames even when they are not exactly the same. It can not only find filenames that are the same, but it also finds similar filenames.
dupeGuru Dulicate Remover
Find Duplicate Songs
The app has a special Music mode that can scan tags and shows music-specific information in the duplicate results window. Searching for similar songs is not possible with this tool. It looks only at ID3 tags. If you want to deduplicate audio files then look at the dedicated music duplicate finders in the list above.
Find Duplicate Pictures
According to the author, the program has a special Picture mode that can scan pictures fuzzily. That will let you find pictures that are similar, but not exactly the same. Unfortunately, such a mode did not detect any similar photos. The author does not provide any details on how such a mode works. Obviously, it does not analyze the photos to find similarities. The term fuzzy is usually used to describe a basic method for finding partially matching items. Obviously, dupeGuru is not an exception to this rule.
Please note that those photo and music similarity features are way worse than any dedicated cloned pictures or song removers on this list. If you want to organize photos or music then go for the dedicated tools at the top of our list.
You can tweak the matching engine to find exactly the kind of duplicates you want to find. Change the scanning engine on the preferences page. Those engines have been designed with safety in mind. There is a reference directory system, as well as a grouping system. I did not find how those work though. They will prevent you from deleting files that you don’t mean to delete.
Removing Cloned Files
Do whatever you want with your duplicates. Not only can you delete duplicate files dupeGuru finds, but you can also move or copy them elsewhere. There are also multiple ways to filter and sort your results to easily weed out false duplicates (for low threshold scans).
Supported languages: English, French, German, Chinese (Simplified), Czech, Italian, Armenian, Russian, Ukrainian, Brazilian, and Vietnamese.
dupeGuru Verdict
If you want to experiment with this tool then please do it in a test folder. It receives a higher ranking in our top tools to delete duplicates only because it supports multiple operating systems and is free. Overall the tool is very basic and does not provide many options. The scan results are a bit questionable. I would not recommend you to use it if you are working on important tasks.
Download dupeGuru
10. Duplicate File Finder Plus
Rating: Average
Duplicate File Finder Plus is another duplicate remover app on our list of the Top duplicate file finders, but it does not impress with anything fancy or better than others. Actually, the tool is rather basic and slow, but it claims to provide an option to compare file content. The developers have not made any effort to hire someone with a good command of English to build their website. This is rather concerning – I personally would not trust my files to be managed by such a tool. Take a look at this paragraph for example:
Compare File Contents, 100% Accurate Result
Compare whether files are the same at the binary level, that is, compare file contents. Since only the file contents are compared, the truly duplicate files must be found, regardless of whether the file names and modification times are consistent, even the extensions are inconsistent (considering the case of deliberately modifying the extensions for safety), they can be found out also.
It is hard to even understand what that text means. Let’s leave aside the website and help files. Those are obviously poorly written and hard to understand. Let’s see what the tool has to offer instead.
Duplicate File Finder Plus Features
The user interface for finding duplicates starts with a screen for selecting the source drives and folders. The interesting thing is that you can add also files – that is not something that we see often. Here you can also include or exclude files or folders.
There are options to search certain file categories – Audio, Video, Documents, Images, Applications, and Compressed. You can exclude or include files by extension too.
Filtering by size is limited to size ranges though and you can not set a range manually. Duplicates can be searched in ranges of Tiny 1 byte – 16 KB, Small 16KB – 1 MB, Medium 1-128MB, and so on.
The only other item in this view is the “Find Now” button that starts the cloned files search.
Duplicate File Finder Plus Results
The duplicate list is really basic and cluttered – see the screenshot below.
Duplicate File Finder Plus
The items are really hard to read and the lack of preview makes the process of reviewing duplicates a real pan. There are not many options and the options to filter the items by size are permanently present in this view again. That is rather strange as such a feature should not be something that is used too often. There are some options to select duplicates automatically on the toolbar.
There are no options to move or copy the duplicates – you can only delete them. You can save the duplicates list to a file though.
Overall the tool is basic and clumsy. To make things worse the price is subscription-based. 30 USD per year for a duplicate finder is a rather steep price. For comparison, the tools at the top of our list cost about the same price for a LIFETIME LICENSE.
Download Duplicate File Finder Plus: https://duplicatefilefinder4pc.com/duplicate-file-finder-plus.htm
11. AntiDupl
Rating: Average
Many computer users have large collections of images in various formats. Certainly, most of these collections have a large number of duplicates. Of course, no one wants duplicates and it is quite normal to want and try to get rid of them. However, if the collection is large enough, doing this manually is very tedious and unproductive work. AntiDupl.NET program will help you automate this process. It can find and display duplicate images in the main graphic formats: JPEG, GIF, TIFF, BMP, PNG, EMF, WMF, EXIF, ICON, JP2, PSD, DDS, and TGA. The comparison is based on the contents of the files, so the program can find not only almost identical but similar images. In addition, the program can find images with some types of defects.
AntiDupl
AntiDupl.NET program is free and open-source software. It is simple to use, has high speed and accuracy of work, and supports Russian and English interfaces.
AntiDupl Verdict
This is a rather outdated tool that seems to be discontinued. As you can see from the screenshot the tool is quite basic. It is free, but do not expect any miracles from it. You can try it on your photo library, but use the preview option to inspect the results. Make sure that you are not deleting anything important.
Download AntiDupl
12. CloneSpy
Rating: Average
If you have an older system like Windows XP CloneSpy could be good for you. It is a rather outdated and old application. The last update on the website read:
Latest version: 3.43 (released on Saturday 12.10.2019)
You can tell that the tool is quite old by the description which intensively references duplicate files on DVDs and also by the user interface. The product is not intuitive and the options are poorly named. For example “Some scanning properties (See options)”, “Really delete files”, “Override delete selection”, etc. To top that up the user interface is totally different than other duplicate search apps.
CloneSpy User Interface
As mentioned the three list boxes on the left are something that you will not find in another duplicate finder. Those are the Pool1, Pool2, and Checksum (CSC) files. They have options to add/remove folders and files and also to lock the lists. Most likely CSC stands for CloneSpy Checksum and the tool uses checksums to compare files. As I have already mentioned, using checksums is fast, but not reliable. Sometimes completely different files can have the same checksum. It is scary to see a 5 kilobyte file identified as a duplicate of a 5 gigabyte one.
On the right side, you can choose which pools (lists) to compare – Pool 1, Pool 2, Both Pools, Pool 1 + CSC, Pool 2+ CSC. The developer most likely chose those unconventional pools to make it possible to compare folders by adding them to different pools. Next to the pool modes are the scan methods. Below them, you will find the delete options and some scanning options information pulled from the settings.
CloneSpy Duplicates Cleaner
CloneSpy Duplicate Search Methods
In the “Search for” panel you will find the options:
- Duplicates – search for duplicates using CSC (custom CRC).
- Duplicates + same name – search for duplicates that also have the same names.
- Same name – search for files with the same names.
- Same name + (approx,) same size – search for files that have the same name and slight difference in size.
- zero bytes long – find empty files
Most likely this quote from the product website refers to the last scan method:
“CloneSpy is able to find files that are not exactly identical, but have the same file name. Perhaps you have different versions of a file and you want to find all of them and remove the older versions. CloneSpy can also find zero length files. These are files that have no content.”
It is not clear what the CSC checksum is and how reliable it is. Using a custom checksum is even more unreliable than using popular well-established algorithms.
CloneSpy Select and Delete Actions
There are no manual dupe selection options which is a big disadvantage. You have the option to either delete all the files automatically or delete them one by one. The delete options that you have are:
- Automatically delete (without confirmation)
- Only in Pool 1
- Only in Pool2
- Older Files
- Newer Files
- Smaller Files
- Larger Files
- Files with shorter file names
- Files with longer file names
- Export a list of all equal files
- Ask user
An option to delete directly or defer the deletion to a batch file is also available. Exporting a list of equal files is a useful feature for advanced users.
CloneSpy Verdict
Overall, CloneSpy is an outdated and unconventional duplicate remover, yet it has its pros and cons.
Pros
- Unconventional approach to finding duplicate
- Export duplicates list
- Defer deletion to bat files
- Find duplicates of Hash lists
- Support for ancient operating systems like Windows 2000 and XP
- Free
Cons
- Outdated and unintuitive user interface
- Unreliable matching algorithm that relies on checksums
- Not updated more than 4 years – probably discontinued
While I would not recommend it to novice users it might be useful for certain deduplication needs. Nevertheless, if you decide to give it a try do so on a test folder first. The user interface is beyond unintuitive and it is not clear how features work out of the box. The descriptions are quite ambiguous and you should experiment to understand how features behave.
Download CloneSpy
13. Deduplicator
Rating: Average
Find, Sort, Filter & Delete duplicate files with Deduplicator. It is a command line duplicate remover that supports Windows, Mac OS, and Linux. If you are familiar with CMD or consoles this is the duplicate finder for you.
The tool is rather nice for a console application, but unfortunately, it uses checksums to identify duplicates. As we already know those are not reliable to identify repeating files.
It looks quite nice compared to other console applications and provides nice progress indication and quite some features. There are even options to filter what types of duplicates to search. You can filter by file type, minimum and maximum file size, and even depth. In addition to those, the tool can follow also symlinks and hard links.
If you are not familiar with console applications you should look at the rest of the tools in our comprehensive list of the best duplicate finders. Consoles are powerful and can automate a lot of tasks, but are prone to errors. If you are not careful you may easily end up deleting the wrong files.
Deduplicator
Usage
Usage: deduplicator [OPTIONS] [scan_dir_path]
Arguments:
[scan_dir_path] Run Deduplicator on dir different from pwd (e.g., ~/Pictures )
Options:
-t, --types Filetypes to deduplicate [default = all]
-i, --interactive Delete files interactively
-s, --min-size Minimum filesize of duplicates to scan (e.g., 100B/1K/2M/3G/4T) [default: 1b]
-d, --max-depth Max Depth to scan while looking for duplicates
--min-depth Min Depth to scan while looking for duplicates
-f, --follow-links Follow links while scanning directories
-h, --help Print help information
-V, --version Print version information
--json
Examples
# Scan for duplicates recursively from the current dir, only look for png, jpg & pdf file types & interactively delete files
deduplicator -t pdf,jpg,png -i
# Scan for duplicates recursively from the ~/Pictures dir, only look for png, jpeg, jpg & pdf file types & interactively delete files
deduplicator ~/Pictures/ -t png,jpeg,jpg,pdf -i
# Scan for duplicates in the ~/Pictures without recursing into subdirectories
deduplicator ~/Pictures --max-depth 0
# look for duplicates in the ~/.config directory while also recursing into symbolic link paths
deduplicator ~/.config --follow-links
# scan for duplicates that are greater than 100mb in the ~/Media directory
deduplicator ~/Media --min-size 100mb
Windows (Pre-built Binary)
You can download the pre-build binary from the Releases page. download the deduplicator-x86_64-pc-windows-msvc.zip zip file for windows. unzip the zip file & move the deduplicator.exe to a location in the PATH system environment variable.
Note: If you Run into an msvc error, please install MSCV from here
Performance
Deduplicator uses size comparison and fxhash (a non non-cryptographic hashing algo) to quickly scan through large number of files to find duplicates. its also highly parallel (uses rayon and dashmap). I was able to scan through 120GB of files (Videos, PDFs, Images) in ~300ms. checkout the benchmarks
Download: Deduplicator
14. Wise Duplicate Finder
Rating: Basic
Wise Duplicate Finder is another clone remover in our list of the best duplicate file finders. Here is what the developer promises:
Find and Delete Duplicate Files on Windows
A duplicate file management tool that can help you find and delete duplicate files by comparing file names, file size, or contents. Get rid of annoying duplicates and free up more space on your hard disks.
Wise Duplicate Finder
Handle Various Types of Duplicates
In our daily routines, we often accumulate numerous documents, photos, songs, and movies, gradually consuming our hard drive space. Over time, this accumulation can lead to disk space shortages. Wise Duplicate Finder offers a solution by identifying and eliminating unnecessary duplicates, thereby freeing up valuable disk space. With Wise Duplicate Finder, you have the flexibility to manage duplicates of any type, whether they are duplicate photos, videos, documents, or more.
Discover Empty Files
Following a cleanup of your hard drive, you might discover numerous empty files or zero-sized files that you wish to remove. Instead of tediously locating them manually, wouldn’t it be more convenient to employ a tool that can swiftly identify and delete them for you?
Manually or Automatically Remove Duplicate Copies
Wise Duplicate Finder serves not just as a duplicate file locator but also as a duplicate file remover. Upon completion of the scanning process, the software displays all duplicated files in categorized groups. It grants you complete authority to eliminate duplicates. You can review the search results and determine which copies to erase. However, sifting through a large volume of duplicates manually can be time-consuming. Hence, you can opt for the program to automatically retain one copy.
Backup & Restore
Never fear accidental file deletions again. Should you mistakenly delete files, you can easily retrieve them using the restore feature. With Wise Duplicate Cleaner, restoring them is just a click away.
Our Review of Wise Duplicate Finder
The very first time you look at the Wise Duplicate Finder website, you can understand that the tool does not live up to its name. The developer starts describing how duplicates are accumulated rather than pointing out the advantages of his too. Then he points out that the tool can find all kinds of duplicate files which is something common for any general-purpose duplicate finder. Next comes the surprising paragraph named “Find Empty Files” which contains an even more confusing description:
After cleaning out files on your hard drive, you may be left with a bunch of empty files or zero size files you want to delete. Rather than finding them manually, wouldn’t it be easier to use a tool that can find them for you and delete them?
Empty or zero-size files have absolutely nothing to do with finding duplicates. The developer claims that you may be left with a “bunch of empty files” after using his tool which is really concerning. To be honest, if I were looking for a duplicate remover tool and saw such a statement I would immediately skip it.
Unfortunately, that is not an exception and there are many similar ambiguous claims all over the website. For example “Wise Duplicate Finder is not only a duplicate file finder but also a duplicate file remover.”. This totally makes no sense as the main purpose of all duplicate finders is to remove the found duplicates.
Wise Duplicate Finder Features
There is no information on the duplicate search method. The tool probably relies on checksums as the majority of the basic duplicate finders on the Internet. Looking at the user guide we see the text: “Compare mode has 3 options, Match name and size, Partial match, and Exact match. The default is Match name and size. Different ways have different results. Of course, it will also take a different time.”
Looking at the file name and size to identify duplicates is not a reliable method. What is partial match and exact match is not clarified.
Leaving the ambiguous scan methods aside the tool is rather basic. The user interface uses an outdated Office 2000 style and does not match the modern Windows UI. It is lacking major features like integrated preview and as we know it is not wise to delete files without reviewing them.
Finally, this rather basic and uninspiring is subscription-based and costs $20 yearly without VAT. In contrast, the best duplicate finder in this list has a free version with an integrated preview and its PRO version costs just $40 for a lifetime license.
Overall it is not wise to pay $20 yearly for a basic tool like Wise Duplicate Finder when you can get better features in the free alternatives.
Download: Wise Duplicate Finder
15. SearchMyFiles
Two Star Rating
In general, this tool is designed to search for files on a Windows system. The duplicate finding is an additional feature that it provides.
SearchMyFiles is an alternative to the standard “Search For Files And Folders” module of Windows. It allows you to easily search files in your system by wildcard, by last modified/created/last accessed time, by file attributes, file content (text or binary search), and by file size. SearchMyFiles allows you to make a very accurate search that cannot be done with Windows search. For example, You can search all files created in the last 10 minutes with sizes between 500 and 700 bytes.
After you made a search, you can select one or more files, and save the list into a text, HTML, CSV, or XML file, or copy the list to the clipboard.
SearchMyFiles Duplicate Finder
Duplicate Search Mode
Starting from SearchMyFiles v1.35 you can search for duplicate files on your system. In order to use this feature, simply change the search mode in the ‘Search Options’ window to ‘Duplicates Search’.
If you switch to ‘Duplicates Search’ mode, all other search options are still relevant. SearchMyFiles will search for duplicates according to these settings. For example, if you choose to search for files larger than 100KB, the tool will list only the duplicates that are larger than that size. In fact, it’s highly recommended to narrow down your duplicate search by using the filtering options. If you don’t do it, the duplicate search will be very slow, and it’ll consume a large amount of memory.
Duplicate Number Column
When you are in ‘Duplicates Search’ mode, SearchMyFiles will first scan all files and folders according to your settings. Then it will locate the duplicate files and add them to the files list on the main window. SearchMyFiles automatically marks duplicate files with identical colors. There is also a ‘Duplicate Number’ column, which displays an identical number for each chunk of duplicate files. You can click the ‘Duplicate Number’ column header in order to sort by this column and to easily view the chunks of duplicate files.
Be aware that the duplicate search is done by making a binary comparison of the files of the same size, byte by byte.
This tool is portable and does not require installation. You can just download and run it. It will not modify the registry and it will not write to your system folders.
SearchMyFiles Verdict
Overall it is a simple tool with quite an outdated user interface. It does not include a preview which makes the duplicate files review process really difficult, but you get what you pay for. The tool is absolutely free, so if you need to perform simple deduplication on fewer files it can get the job done. If you need something more advanced you should consider some of the tools at the top of this best duplicate file finders list. There are other free duplicate finder alternatives there that include preview and other handy features. Nevertheless, it is a free tool that is worth mentioning in our list of the best duplicate file finder applications.
Download SearchMyFiles
16. SourceForge Duplicate Files Finder
Rating: Poor
Duplicate Files Finder is another multi-platform free duplicate cleaner. Duplicate Files Finder is a cross-platform application for finding and removing duplicate files by deleting, creating hardlinks, or creating symbolic links. A special algorithm minimizes the amount of data read from the disk, so the program is very fast.
SourceForge Duplicate Files Finder Features
This tool is rather basic and really outdated. As with most such tools, the duplicate removal process starts by selecting folders to be scanned and applying search filters. There are per-folder filtering options, which is nice. You can include or exclude the subfolders in the scan and also filter by file name. In addition to that, you can filter files by minimum and maximum sizes. Options to filter hidden or empty files are available too.
Though there are some features, the user interface is poor and difficult to use. For example, you have to filter file names using wildcards. This is not something that a common user is familiar with. Filtering files by size accepts values only in bytes. If you want to specify sizes that are gigabytes large you have to type huge numbers.
If you are a computer guru in an adventurous mood you may want to play with this tool as it is free. Getting even the basic deduplication task completed using it, is a challenge.
SourceForge Duplicate Files Finder
SourceForge Duplicate Files Finder 0.8.0 Changes (2008-06-01 )
This release mainly adds symbolic link support for Windows, many GUI changes, improved search support, and improved compiling support.
The changes in detail:
– made the build more tolerant of different build environments
– added symbolic link support for Windows
– improved search options (masks with ‘;’ (windows) or ‘:’ (Unix) in it are allowed; exclude masks were introduced)
– many GUI changes
– created DLL with common code
– program icon for Windows
– progress display improved
– and many other changes
SourceForge Duplicate Files Finder Verdict
I don’t recommend using this tool unless you are a computer expert. It is very difficult to use and the last update dates back to 2008. This tool is in our most useful repeating file finders list just for the sake of completeness.
Download SourceForge Duplicate Files Finder: https://sourceforge.net/projects/doubles/
Best Duplicate File Finder Summary
This is our list of the top tools to remove duplicates. They are sorted from best to worst. Choosing the best duplicate file finder depends on your file deduplication tasks. Nevertheless, you can’t go wrong with any of the first 7 tools above. Certainly, if you download any of the top 3 tools in the list, I am quite sure that they will cover all your requirements.
It is best to keep a well-organized file system that is free of duplicates. Surely, those just waste space and clutter the system by increasing the count of files. Nevertheless, you should be very careful before deleting any files. Therefore, if you are not an expert you should try the tools on a test folder or a test machine first. In addition to that, if yodu are not sure how a feature works, you should read the help file instead of experimenting with your valuable data.
Please share this best duplicate file finder list with your friends – they will thank you as it contains a lot of useful and free tools.
Pingback: Best Duplicate Photo Finder & Removers in 2022 - Top 17 Image Cleaners
Pingback: Find Largest Files in Windows 11, 10, 8, 7
Very complete list and great in-depth review. I tried some of the tools and totally agree with you.