How do I keep the first n lines of a file/command, but grep the rest?

user_1394353 0
Viewed 88

Easiest to give an example.

bash-$ psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;


        relname         | reltype
------------------------+---------
 bme_reltag_02          |       0
 bme_reltag_type1_type2 |       0
 bme_reltag_10          |       0
 bme_reltag_11          |       0
 bme_reltag_cvalue3     |       0   what I care about

But what I am really interested in is anything with cvalue in it. Rather than modifying each query by hand (yes, I know I could do it), I can egrep what I care about.

psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;' | egrep 'cvalue'

but that strips out the first two lines with the column headers.

 bme_reltag_cvalue3     |       0

I know I can also do this:

psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;' | head -2 && psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;' | egrep 'cvalue'

        relname         | reltype
------------------------+---------
 bme_reltag_cvalue3     |       0

but what I really want to do is to keep the head (or tail) of some lines one way and then process the rest another.

My particular use case here is grepping the contents of arbitrary psql selects, but I'm curious as to what bash capabilities are in this domain.

I've done this before by writing to a temp file and then processing the temp file in multiple steps, but that's not what I am looking for.

filtering sed awk
4 Answers

A while read loop and grep, if that is acceptable.

#!/usr/bin/env bash

while IFS= read -r lines; do
  [[ $lines == [12]* ]] && echo "${lines#*:}"
  [[ $lines == *cvalue[0-9]* ]] && echo "${lines#*:}"
done < <(psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;' | grep -n .)

Without the grep an alternative is a counter to know the line number, which will be a pure bash solution.

#!/usr/bin/env bash

counter=1
while IFS= read -r lines; do
  [[ $counter == [12] ]] && echo "$lines"
  [[ $lines == *cvalue[0-9]* ]] && echo "$lines"
  ((counter++))
done < <(psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;')

If bash4+ is available.

#!/usr/bin/env bash

mapfile -t files < <(psql -c 'select relname, reltype from pg_catalog.pg_class limit 5;')

printf '%s\n' "${files[0]}" "${files[1]}"

unset 'files[0]'  'files[1]'

for file in "${files[@]}"; do
  [[ $file == *cvalue[0-9]* ]] && echo "$file"
done

  • By default the builtin read strips the leading and trailing white spaces, so in this case we don't want that, so we use IFS=

  • grep -n . adds the line number with a :

  • [12] is a glob not regex which means either 1 or 2 and the glob * will match if it is the first character of the line.

  • *cvalue[0-9]* will match cvalue and any amount of int/digit next to it.

  • "${lines#*:}" is a parameter expansion that strips the leading :

  • <( ) is called process substitution.

user_44522650 
$ psql -c ... | awk 'NR<3 || /cvalue/' file
user_14358690

This can be done with sed using its range feature to only operate on lines 3 and beyond

sed '3,${/cvalue/!{d;};}'

Proof of Concept

$ cat ./psql
relname                | reltype
------------------------+---------
bme_reltag_02          |       0
bme_reltag_type1_type2 |       0
bme_reltag_10          |       0
bme_reltag_11          |       0
bme_reltag_cvalue3     |       0

$ sed '3,${/cvalue/!{d;};}' ./psql
relname                | reltype
------------------------+---------
bme_reltag_cvalue3     |       0

Explanation

  • 3,${...;}: Start processing from line 3 until the end of file $
  • /cvalue/!{d;}: Delete d any line that does not match (!) the regex /cvalue/
user_2072480

You can use bash.. tail.and head commands

cat file.sql | head -n 15 > head.sql

Replace the 15 with the number of lines

Or replace head with tail... for the bottom of the file

user_98851790
Related