Chain awk regex matches like grep

user_17157197 0
Viewed 368

I am trying to use awk to select/remove data based on cell entries in a CSV file.

How do I chain Awk commands to build up complex searches like I have done with grep? I plan to use Awk to select rows based on matching criteria in cells in multiple columns, not just the first column as in this example.

Test data

123,line1
123a,line2
abc,line3
G-123,line4
G-123a,line5

Separate Awk statements with intermediate files

awk '$1 !~ /^[[:digit:]]/ {print $0}' file.txt > output1.txt
awk '$1 !~ /^G-[[:digit:]]/ {print $0}'  output1.txt > output2.txt
mv output2.txt output.txt
cat output.txt

Chained or multi-line grep version (I think limited to first column only)

grep -v \
     -e "^[[:digit:]]" \
     -e "^G-[[:digit:]]" \
     file.txt > output.txt
cat output.txt

How can I rewrite the Awk command to avoid the intermediate files?

awk
6 Answers

In your awk commands and example, awk regards file.txt as having only one field because you have not defined FS, so the default whitespace field separator is used.

With that said, you can easily AND your two pattern matches together like this:

awk '($1 !~ /^[[:digit:]]/) && ($1 !~ /^G-[[:digit:]]/) {print $0}' file.txt

To make awk use comma as a field separator, you can define it in a BEGIN block. In this example, the output should be just line3

awk 'BEGIN {FS=","} ($1 !~ /^[[:digit:]]/) && ($1 !~ /^G-[[:digit:]]/) {print $2}' file.txt
user_37412190

Generally, in awk there are boolean operators available (it's better than grep! :) )

awk '/match1/ || /match2/' file
awk '(/match1/ || /match2/ ) && /match3/' file

and so on ...

In your example you could use something like:

awk -F, '$1 ~ /^[[:digit:]]/ || $1 ~ /G-[[:digit:]]/' input >> output

Note: This is just an example of how to use boolean operators. Also the regular expression itself could have been used here to express the alternative match:

awk -F, '$1 ~ /^(G-)?[[:digit:]]/' input >> ouput
user_1713180

I would suggest the literal translation of that grep command in awk is

awk '
  /^[[:digit:]]/ {next}
  /^G-[[:digit:]]/ {next}
  {print}
' file.txt

But you have several examples of how to write it more concisely.

user_75520

You can use

awk '$1 !~ /^(G-)?[[:digit:]]/' file.txt > output.txt

The awk tries to find in Field 1:

  • ^ - start of string
  • (G-)? - an optional G- char sequence (note the regex flavor in awk is POSIX ERE, so (...) denotes a capturing group and ? denotes a one or zero times quantifier)
  • [[:digit:]] - a digit.

If the match is found, the record (=line) is not printed. Else, the line is printed.

user_38329700

to stick to your question, I would use:

awk '$1 !~ /^[[:digit:]]/ && $1 !~ /G-[[:digit:]]/' file.txt > output.txt

But I like the @Wiktor Stribiżew REGEX approach!

user_34151590

With your shown samples, this could be also done in grep in a single regexp, we need not to chain the different regex, adding this solution in case you/anyone need it; could be helpful.

grep -v -E '^(G-)?[[:digit:]]' Input_file

Explanation: Simple explanation would be, using grep's -v option to omit lines which are matching the mentioned pattern. Then using -E option of it to enable ERE(extended regular expressions). In main program using regex ^(G-)?[[:digit:]] to match if line starts from G- OR digit then don't print that line.

user_58665800
Related