Removing duplicate rows in Notepad++

user_471178 0
Viewed 814529

Is it possible to remove duplicated rows in Notepad++, leaving only a single occurrence of a line?

duplicates notepad++
17 Answers

In version 7.8, you can accomplish this without any plugins - Edit -> Line Operations -> Remove Consecutive Duplicate Lines. You will have to sort the file to place duplicate lines in consecutive order before this works, but it does work like a charm.

Sorting options are available under Edit -> Line Operations -> Sort By ...

user_8251600

As of Notepad++ version 8.1, there is a specific command to do precisely what this popular question asks. On can remove duplicated rows in a text file with the menu command Edit > Line Operations > Remove Duplicate Lines.

There is no need to install a plugin (as the currently accepted answer suggests), or sort the lines beforehand, or use the regex syntax in the Replace dialogue as other answers suggested.

enter image description here

user_37538260

As of now, it's possible to remove all consecutive duplicate lines with Notepad in-built functionality. Sort the lines first:

Edit > Line Operations > "Sort lines lexicographically",

then

Edit > Line Operations > "Remove Consecutive Duplicate Lines".

The regex solution suggested above didn't remove all duplicate lines for me, but just the consecutive ones as well.

user_60774260

You may need a plugin to do this. You can try the command line cc.ddl(delete duplicate lines) of ConyEdit. It is a cross-editor plugin for the text editors, including Notepad++.

With ConyEdit running in background, follow the steps below:

  1. enter the command line cc.ddl at the end of the text.
  2. copy the text and the command line.
  3. paste, then you will see what you want.

Example
enter image description here

user_99185100

Whether the file is sorted or not, you can use below regex to remove duplicates in anywhere occurred in your file.

Find what: ^([^\r]*[^\n])(.*?)\r?\n\1$
Replace with: \1\2
Search Mode:

  • "Regular expression"
  • Check the ". matches newline" option

Click "Replace All" as many time as possible (or press and hold Alt+A shortcut key) until you see "0 occurrences were replaced"

user_40239500

Extending the top answer, you can also use a 2nd lookahead to find rows that are almost duplicates of other rows.

^(\s*(<PackageReference Include=".*" Version=).*)$\s+?^(?=.*^\2.*$)

Here I'm after multiple references to the same <PackageReference Include=".*" string, regardless of its version.

Test data

<PackageReference Include="Package1" Version="2.2.1" />

    <PackageReference Include="Package1" Version="2.2.1" /> // Match
<PackageReference Include="Package1" Version="2.2.2" />

<PackageReference Include="Package2" Version="5.1" /> // Match
<PackageReference Include="Package2" Version="5.2" />

<PackageReference Include="Package3" Version="2.2.1" /> // No match
<PackageReference Include="Package4" Version="2.2.1" />

See a breakdown of what the regex terms mean and try with your own data on this regex101 share.

BlueNovember_284110

Difficult to do this in NPP. Better way is following:

Download cygwin utility, it is simple Linux terminal under windows. It allow to execute any Linux command in Windows. And you have sort -u there.

user_51634520

Notepad++ has builtin operations:

Edit -> Line Operations -> Sort Lines...
Edit -> Line Operations -> Remove Duplicate Lines

Maybe it works with just Remove Duplicate Lines, but I needed to see that the operations work by seeing that sorting works.

If it doesn't work, problem could be with different ending of lines, which I encountered now. You can check it with View -> Show Symbol -> Show End of Line. Replace it to have it same.

Click on Search > Replace (or Ctrl + H)
Find what:  \r\n
Replace with: \n
Search Mode: select Extended (\n, \r,...)
Replace All
user_34872930
Related