Remove duplicates
This software analyses text files and remove lines that appears more than one time.
remove duplicates v 0.6 keeps the first line on top. If your file do not have a header, please insert a comment as first line.
Manual :
1- if not already installed, install Perl free programming language
2- unzip the software
3- copy data text (or csv) files in the “data” directory
4- execute the software by the command : perl software_name or double click on the .pl file
5- processed files are in the “results” directory.
6- The number of duplicates are in the "counts" directory
7- details are displayed in the duplicate.log file.