Sed to remove substring | Can I make a flexible pattern to remove numbers before tab?

user_14253755 0
Viewed 72

I wanted to ask some advice on an issue that I'm having in removing a substring from a string. I have a file with many lines like the following:

    DOG; CSQ| 0.1234 | abcd | \t CAT

where \t represents a literal tab.

My aim is to remove a substring by using sed 's/CSQ.*|//g' so that I can get the following output:

    DOG; CAT

However I face a problem where all the rows aren't formatted the same. For example, I also get lines such as:

    DOG; CSQ| 0.1234 | abcd | 0 \t CAT
    DOG; CSQ| 0.1234 | abcd | 0.9187 \t CAT

My code fails at this point because instead of getting DOG; CAT for all lines, I get:

    DOG; CAT    
    DOG; 0 CAT
    DOG; 0.9187 CAT

I've searched for possible solutions but I'm having difficulty (I'm also quite new to bash). I imagine there's something that I can do with sed that will handles all cases but I'm not sure.

sed bash
3 Answers

You can find and replace all text from CSQ till the last | and all chars after that till the tab including it using

sed 's/CSQ.*|.*\t//' file > newfile

See the online demo.

The CSQ.*|.*\t is a POSIX BRE pattern that matches

  • CSQ - a CSQ string
  • .* - any text
  • | - a pipe char
  • .* - any text
  • \t - TAB char.

If the \t are two-char combinations double the backslash before t:

sed 's/CSQ.*|.*\\t//' file > newfile

See this online demo.

user_38329700

So optionally match it.

sed 's/CSQ.*|\( [0-9.]*\)\?//g'

You can learn regex online with fun with regex crosswords.

user_90727530

awk makes this pretty easy.

$: awk '/CSQ.*\t/{print $1" "$NF}' file
DOG; CAT
DOG; CAT
DOG; CAT

Note that the file has to have actual tabs, not \t sequences. awk will read the \t correctly.

If there are no other formatted lines in the file that you want, then maybe just

$: awk '{print $1" "$NF}' file
DOG; CAT
DOG; CAT
DOG; CAT
user_86565520
Related