Remove multiple patterns of a file

user_13472757 0
Viewed 121

I've got the following file, lets call it user.txt:

2020/06/30 22:02:58 > [+] VALID USERNAME:   admin@spookysec.local
2020/06/30 22:03:58 > [+] VALID USERNAME:   admin-svc@spookysec.local
2020/06/30 22:04:58 > [+] VALID USERNAME:   backup@spookysec.local
2020/06/30 22:05:58 > [+] VALID USERNAME:   harry@spookysec.local
2020/06/30 22:06:58 > [+] VALID USERNAME:   wrgwrsd@spookysec.local
2020/06/30 22:07:58 > [+] VALID USERNAME:   brssdb@spookysec.local
2020/06/30 22:09:58 > [+] VALID USERNAME:   a@spookysec.local
2020/06/30 22:10:58 > [+] VALID USERNAME:   adm@spookysec.local
2020/06/30 22:11:58 > [+] VALID USERNAME:   admifdfdsn@spookysec.local

I want to remove multiple patterns of every line to just get the usernames (e.g. admin, admin-sv) with one username per line. I tried two seperate sed's what worked fine,

sed -r 's/.{16}$//' user.txt > user2.txt

and after that

sed -r 's/.{42}//' user2.txt > user3.txt

but I think there is a faster and sexier way to do that.

Especially this command

sed -r 's/(@\D+)$//' user.txt > user2.txt

doesn't work. I wanted to delete the @ and every character hereafter...

Does anyone have a clue what I did wrong and possibly a solution? It would be best if I can reach my goal with sed or grep.

sed grep awk
5 Answers

You may try this sed:

sed -E 's/.*[[:blank:]]|@.*//g' file

admin
admin-svc
backup
harry
wrgwrsd
brssdb
a
adm
admifdfdsn

Explanation:

  • .*[[:blank:]]: Match any string of 0 or more characters followed by a whitespace
  • |: OR
  • @.*: Match @ and everything after that
user_5482250

With your shown samples, please try following codes.

1st solution: With awk, this could be done in more simpler way, try following once.

awk -F'VALID USERNAME:[[:space:]]+|@' '{print $2}' Input_file

Explanation: Using awk programing here. Making VALID USERNAME:[[:space:]]+|@ as field separator for whole Input_file and printing 2nd field of each line as per requirement.

2nd solution: Using GNU grep please try following.

grep -oP 'VALID USERNAME:[[:space:]]+\K[^@]*' Input_file

Explanation: using GNU grep's -oP option where -o to print only matched value and -P to enable PCRE regex. In main program of grep matching VALID USERNAME:[[:space:]]+ regex followed by a \K(which will forget previously matched values in order to not to print them), then matching everything just before @ to get actual user names as per OP's shown samples.

user_58665800

Another variant using sed with a capture group, using group 1 \1 in the replacement:

sed -E 's/.*[[:blank:]]([^@[:blank:]]+)@.*/\1/' file
  • .*[[:blank:]] Match until the last space
  • ([^@[:blank:]]+) Capture in group 1 matching 1+ chars other than @ or spaces
  • @.* Match @ and the rest of the line

Output

admin
admin-svc
backup
harry
wrgwrsd
brssdb
a
adm
admifdfdsn

Another option using awk, you could split the last column on @

If the split has returned 2 parts, you can print the first part:

awk '{
    nr=split($(NF), a, "@")
    if (nr==2) print a[1]
}' file
user_54249880

I would use:

sed 's/.*[[:space:]]\([^[:space:]]\)/\1/; s/@.*//'

The slight difference between this and anubhava's (good) answer, is just that it still works if there is stray white space at the end of a line.

Possibly unnecessary/paranoid, but depending on the situation, I like to allow for stuff like that.

If the columns vary, you could use:

sed -E 's/(.*VALID[[:space:]]+USERNAME:[[:space:]]+)([^@[:space:]]+)(.*)/\2/'

This matches VALID USERNAME: anywhere in the line, and prints the next field, up to but not including the first @ or whitespace character. You could also change the pattern to just USERNAME: or similar, depending on your data.

Finally, these variations skip a line if @ is not found (ie. the email field is empty or invalid):

sed -n 's/.*[[:blank:]]\([^[:space:]]\)/\1/; s/@.*//p'
sed -En 's/(.*VALID[[:space:]]+USERNAME:[[:space:]]+)([^@]+)(@.*)/\2/p'
user_139196680

This might work for you (GNU sed):

sed -nE 's/.*\<([^@]*)@.*/\1/p' file

Capture and print the word before the last @ character.

user_9674920
Related