How-to: how to remove duplicate files in Linux

duplicate file

When we work with heterogeneous systems with files shared between Windows and Unix machines, it is possible that some operating systems generate a large number of files that are not necessary or practical in a * nix system, such as the famous Thumbs.db among others, which are used to index multimedia content on these systems. On the other hand, in our distro they can be annoying and what is worse, take up storage space that we could be taking advantage of for other purposes.

If in addition to this we have files or duplicate files Whether it is because we have made copies without realizing it or for any other reason, the amount of storage space on our hard disk occupied unnecessarily can be quite high. I have managed to clear some gigs just by eliminating this type of duplicate files. In addition, they tend to be very annoying when we make periodic backups, since they increase the size of the backup and increase the time and space necessary ...

You will already know programs to free up space on our hard disk such as BleachBit, which is capable of eliminating the Thumbs.db that I have mentioned previously, cache, cookies, and other unnecessary files that may be accumulating in our system due to the daily use we make. But it cannot delete duplicate files, however it is possible to do it with the Turkish ones that we give you below:

How to remove duplicates from the console?

As usual, to eliminate duplicate files there are two ways to do it, one is through commands from the terminal as, something that provides flexibility and power without using graphic tools. And if you decide to do the work from the console, one of the best tools to eliminate duplicates that I can present you is FDUPES. It is a fairly common tool and is present in most of the repositories of the known distros, therefore for its installation it will be enough to use the package manager (apt-get, dnf, yum, ...) that we use in each case.

Once installed, so that we detect all duplicate files by inspecting their creation date, content and file names. Imagine you wish find duplicate content in your / home directory, you just have to execute the following:

fdupes /home

If you want to use a recursive search, that is, if it also finds duplicates in its subdirectories, you must use the -r option. With the -S option you can also see the space used by these duplicates. For example:

fdupes -rS /home

Once detected, if we want them delete all, we only use the -d option, which will allow you to remove the ones you want:

fdupes -d /home

For more information you can use man fdupes, since there are more options and possibilities.

How to remove duplicates with graphical tools?

Fslint GUI

On the other hand, many people do not want to make use of the console, and more specifically those that come from more purely graphical environments such as Microsft Windows and MacOS, possibly are a little lost at the beginning. For all of them the solution is to use a graphical tool that allows to locate and eliminate duplicates in an intuitive way, and it exists, it is called FSlint.

As with fdupes, FSlint will be available in most repositories one of the best known distros because it is a fairly popular tool, and of course in the Apps Stores that are usually found in many distros, so its installation does not imply any mystery, simply download and install from the App Store or install it from the console using the package manager you use in each case. Once installed you can start using it.

Su GUI is pretty straightforward as you can see in the image that I have put. Its use is very simple, but for greater ease I will describe the steps of how to use it:

  • At Search Dash, above, you can add or remove directories from your system with the buttons on the right. In the center you will see the ones you already have added. So you will need to add all the paths where you want to find duplicates or if you want it to analyze the entire system add root /. If you notice, on the right side there is a tab that you must mark if the search you want to be done recursively, that is, not only search within that directory but also in the subdirectories it finds.
  • In the central area we have the results menu, with several options in the left column to select what we want to see, as binaries. duplicates, bad links, etc., options that we also found in fdupes. The blank space will be filled in with the results found when you click the Search button.
  • Finally, you can select the files found alone or in a group with the select button and then exert an action on one or a group of them with the Save buttons so that nothing is deleted or Delete so that duplicates are eliminated.

It is certainly a very powerful and simple tool ... I can only comment on the tab Advanced search parametersIf you press it, you will see a new menu where you can select other parameters or filters to exclude certain files from searches or apply specific rules to search for specific files.

Do not forget to leave your comments with doubts, suggestions, etc. I hope this tutorial has helped you ...


Leave a Comment

Your email address will not be published. Required fields are marked with *

*

*

  1. Responsible for the data: AB Internet Networks 2008 SL
  2. Purpose of the data: Control SPAM, comment management.
  3. Legitimation: Your consent
  4. Communication of the data: The data will not be communicated to third parties except by legal obligation.
  5. Data storage: Database hosted by Occentus Networks (EU)
  6. Rights: At any time you can limit, recover and delete your information.