Linux provides various tools to work with text files. The uniq is a command used to detects and deletes duplicate lines in a file. This can be very helpful to remove duplicate text in large text files.
In this article, you see the usage of uniq command along with some examples.
The syntax of the Linux uniq command
The syntax of using uniq command in Linux is given below-
uniq [Option] [Input [Output]]
You find the options that can be used with this command on its man page.
How to remove duplicate lines using uniq command
Suppose we have a file called file1.txt which contains some duplicate lines of text. You can see the content of this file in the given image –
Now to remove all repeated lines use the following command –
You can see the output of this command in the given image –
How to display the number of times a line repeated
By using option -c you can display the number of times a line is repeated. Now execute the following command to display the count of line repetition.
uniq -c file1.txt
You can see the output in the given image –
How to display duplicate lines only
By using option -D with uniq command will print only duplicate lines of text. Use the following command –
uniq -D file1.txt
See the output in the image below –
Similarly, you can use option -u to print non-repetitive lines only.
uniq -u file1.txt
How to make uniq comparison insensitive
By default, uniq command comparison is case sensitive. You can make this command to perform the insensitive comparison by using the option -i. The complete command is given below-
uniq -i file1.txt
To know more about this command you can visit the man page of this command in your terminal by using –
Ok, that’s all for now. Now if you have a query then write us in the comments below.