[ILUG] Remove duplicate lines from a file?

[Lars Hecking]

Conor Daly writes:
> Quick! Quick!  Before everyone stops reading for the weekend!  :-)
> 
> Anyone know of a way to remove duplicate lines from a file using something
> like grep, sed or perl?  I want something l can use in a pipe preferably.
> The duplicates are distributed randomly through the file and I don't want to
> sort if I can avoid it.  Something like
> 
> cat file_containing_masses_of_duplicate_lines_distributed_at_random |
> [duplicate_filter] > file_containg_unique_lines

 Hehehe ... beware of cats :)

> or
> 
> [duplicate_filter]
> file_containing_masses_of_duplicate_lines_distributed_at_random
> file_containg_unique_lines
> 
> Any ideas?
 
 man uniq. Next question.