CoderZone.org

Category: >> Perl Code >> Remove Duplicate Lines from a File Bookmark and Share

<< lastnext >>

Snippet Name: Remove Duplicate Lines from a File

Description: A simple code snippet to remove duplicate lines from a file.

Also see:
» Write or update an htpasswd file i...
» Validate file extensions in upload...
» Javascript Include - allows dynami...
» Remove empty lines from a string o...
» Automatically find location of jav...
» Remove empty lines from text files
» Delete a file in Visual Basic
» Copy a file in VB
» Reading and Writing to text files ...
» Parsing PHP in Javascript Files
» Perl version of file_get_contents&...
» Read the contents of an entire fil...
» Removing duplicate array items whi...
» Create an array from the lines of ...
» Extract Just The BODY Of An HTML F...
» Rename Files Interactively
» Sort A File List By Month Name In ...
» Get File Size Over HTTP
» Get A Directory Listing Of Specifi...
» Check If A Directory Allows File C...
» Copy Files By Date Modified
» Output To STDOUT Or A File
» Include All Files In A Directory
» More File Type Detection in PHP
» Copy a remote file to your site
» Use htpasswd file with PHPscripts
» Check for duplicates and delete du...
» TABLESPACE: List tablespaces, fil...
» TABLESPACE: Tempfile operations

Comment: (none)

Author: CoderZone
Language: PERL
Highlight Mode: PERL
Last Modified: December 03rd, 2010

#!/usr/bin/perl
 
use strict;
use warnings;
 
my $file = '/path/to/file.txt';
my %seen = ();
{
   local @ARGV = ($file);
   local $^I = '.bac';
   while(<>){
      $seen{$_}++;
      next if $seen{$_} > 1;
      print;
   }
}
print "finished processing file.";
 
There haven't been any comments added for this snippet yet. You may add one if you like.  Add a comment 
© coderzone.org | users online: 20