Use sed/awk to remove pattern from repeating lines

user_14678437 0
Viewed 71

I have the following pattern in a file

TREE(abc/x73/APPLE)   DEST(MYFILE.FILE1A) 
TREE(abc/x74/APPLE)   DEST(MYFILE88.FILE1A) 
TREE(abc/x100/APPLE)  DEST(MYFILE100.FILE1B) 
TREE(abc/x61/APPLE)   DEST(MYFILE1000.FILE1A)

I would like the output to ideally be:

TREE(abc/x73/APPLE)   FILE1A
TREE(abc/x74/APPLE)   FILE1A
TREE(abc/x100/APPLE)  FILE1B
TREE(abc/x61/APPLE)   FILE1A

However, I'll be ok with:

TREE(abc/x73/APPLE)   DEST(FILE1A)
TREE(abc/x74/APPLE)   DEST(FILE1A)
TREE(abc/x100/APPLE)  DEST(FILE1B)
TREE(abc/x61/APPLE)   DEST(FILE1A)

I've tried variations of

sed 's/\(FILE1A\).*/\1/'

but I just keep getting the same as the input. I'd appreciate any pointers as to where I'm going wrong.

Thanks.

sed awk
5 Answers

Does this solve your problem?

cat test.txt
TREE(abc/x73/APPLE)   DEST(MYFILE.FILE1A)
TREE(abc/x74/APPLE)   DEST(MYFILE88.FILE1A)
TREE(abc/x100/APPLE)  DEST(MYFILE100.FILE1B)
TREE(abc/x61/APPLE)   DEST(MYFILE1000.FILE1A)

sed -n 's/\(.*)\).*\.\(.*\))/\1\t\2/p' test.txt
TREE(abc/x73/APPLE) FILE1A
TREE(abc/x74/APPLE) FILE1A
TREE(abc/x100/APPLE)    FILE1B
TREE(abc/x61/APPLE) FILE1A

Explanation:

-n option - don't print everything

\(.*)\) first capture group - everything up to and including the first ")"

\.\(.*\)) second capture group - everything between "." and the last ")"

\1\t\2/p print the first captured group, then a tab, then the second capture group

user_129573400 
sed 's/MYFILE[[:digit:]]\{0,\}\.//' file.txt

Output

TREE(abc/x73/APPLE)   DEST(FILE1A)
TREE(abc/x74/APPLE)   DEST(FILE1A)
TREE(abc/x100/APPLE)  DEST(FILE1B)
TREE(abc/x61/APPLE)   DEST(FILE1A)

sed 's/DEST.\{1,\}\.//;s/)$//' file.txt

Output

TREE(abc/x73/APPLE)   FILE1A
TREE(abc/x74/APPLE)   FILE1A
TREE(abc/x100/APPLE)  FILE1B
TREE(abc/x61/APPLE)   FILE1A
user_44522650

I would harness GNU AWK's for this task as follows, let file.txt content be

TREE(abc/x73/APPLE)   DEST(MYFILE.FILE1A) 
TREE(abc/x74/APPLE)   DEST(MYFILE88.FILE1A) 
TREE(abc/x100/APPLE)  DEST(MYFILE100.FILE1B) 
TREE(abc/x61/APPLE)   DEST(MYFILE1000.FILE1A)

then

awk 'BEGIN{FS="DEST\\([[:alpha:][:digit:]]*\\.|\\)[[:space:]]*$"}{print $1 $2}' file.txt

output

TREE(abc/x73/APPLE)   FILE1A
TREE(abc/x74/APPLE)   FILE1A
TREE(abc/x100/APPLE)  FILE1B
TREE(abc/x61/APPLE)   FILE1A

Explanation: I set field seperator (FS) to value which hold two alternatives (sheared by |)

  • DEST( zero or more alphanumerics dot (.)
  • ) followed by zero or more whitespaces (your file has trailing whitespaces) and end of line ($)

As ( and ) and . have special meaning I need to escape them to get literal ( and literal ) and literal . which I need. I print content of value 1st column (which is TREE(...) and trailing whitespaces) concatenated with value of 2nd column which is FILE1A and so on.

(tested in gawk 4.2.1)

user_107859750

Using awk you can get the two results:

First output with gsub() function:

awk '{gsub(/^[^.]+.|)$/,"",$2)} 1' file | column -tc2
TREE(abc/x73/APPLE)   FILE1A
TREE(abc/x74/APPLE)   FILE1A
TREE(abc/x100/APPLE)  FILE1B
TREE(abc/x61/APPLE)   FILE1A

or second output with sub():

awk '{sub(/MYFILE[[:digit:]]*./,"",$2)} 1' file | column -tc2
TREE(abc/x73/APPLE)   DEST(FILE1A)
TREE(abc/x74/APPLE)   DEST(FILE1A)
TREE(abc/x100/APPLE)  DEST(FILE1B)
TREE(abc/x61/APPLE)   DEST(FILE1A)

and using column -tc2 you can keep (or adjust) the spaces between columns.

user_142594650

You might also use a bit more specific pattern, and use a negated character class to match any char except parenthesis between the opening and closing parenthesis.

  • ( Capture group 1
    • \([^()]*\)[[:blank:]][[:blank:]]* Match from the first opening ( till closing ) and the following spaces or tabs
  • ) Close group 1
  • [^()]* Optionally match any char except ( and )
  • \( Match the second opening parenthesis
  • [^()]*\. Match any char except ( and ) and then a dot
  • ([^()]*) Capture any char except ( and ) in group 2
  • \) Match the second closing parenthesis

See a regex demo for the capture groups.

sed -n 's/\(([^()]*)[[:blank:]][[:blank:]]*\)[^()]*([^()]*\.\([^()]*\))/\1\2/p' test.txt

Output

TREE(abc/x73/APPLE)   FILE1A 
TREE(abc/x74/APPLE)   FILE1A 
TREE(abc/x100/APPLE)  FILE1B 
TREE(abc/x61/APPLE)   FILE1A

If the format of the data is always as in the example data, using gnu awk you could also capture the last alphanumeric characters before the closing parenthesis of the second field, and use a tab as an outut field separator.

awk -v OFS='\t' '
match($2, /([[:alnum:]]+)\)/, a){print $1, a[1]}
' test.txt

Output

TREE(abc/x73/APPLE)     FILE1A
TREE(abc/x74/APPLE)     FILE1A
TREE(abc/x100/APPLE)    FILE1B
TREE(abc/x61/APPLE)     FILE1A
user_54249880
Related