[ILUG] Remove duplicate lines from a file?

[Conor Daly]

-----Original Message-----
From: Lars Hecking <lhecking at nmrc.ie>
To: ILUG Mailing List <ilug at linux.ie>
Date: 30 June 2000 15:34
Subject: Re: [ILUG] Remove duplicate lines from a file?


>Conor Daly writes:
>> Quick! Quick!  Before everyone stops reading for the weekend!  :-)
>>
>> Anyone know of a way to remove duplicate lines from a file using
something
>> like grep, sed or perl?  I want something l can use in a pipe preferably.
>> The duplicates are distributed randomly through the file and I don't want
to
>> sort if I can avoid it.  Something like
>>
>> cat file_containing_masses_of_duplicate_lines_distributed_at_random |
>> [duplicate_filter] > file_containg_unique_lines
>
> Hehehe ... beware of cats :)
>
>> or
>>
>> [duplicate_filter]
>> file_containing_masses_of_duplicate_lines_distributed_at_random
>> file_containg_unique_lines
>>
>> Any ideas?
>
> man uniq. Next question.
>

Thanks Lars...

Still have to sort though :-(

But I can live with that...

---
Conor Daly

Ph   +353 1 8326146

conor.daly at oceanfree.net
------------------------------------------