Linux command or script counting duplicated lines in a text file?

user_466309 0
Viewed 103507

If I have a text file with the following conent

red apple
green apple
green apple
orange
orange
orange

Is there a Linux command or script that I can use to get the following result?

1 red apple
2 green apple
3 orange
duplicates linux text
8 Answers

Here is a simple python script using the Counter type. The benefit is that this does not require sorting the file, essentially using zero memory:

import collections
import fileinput
import json

print(json.dumps(collections.Counter(map(str.strip, fileinput.input())), indent=2))

Output:

$ cat filename | python3 script.py
{
  "red apple": 1,
  "green apple": 2,
  "orange": 3
}

or you can use a simple one-liner:

$ cat filename | python3 -c 'print(__import__("json").dumps(__import__("collections").Counter(map(str.strip, __import__("fileinput").input())), indent=2))'
user_34309860
Related