Using sed, extract text between first occurrence of a word1 and last occurrence of a word2

user_5674391 0
Viewed 381

I need to extract text between the first occurrence of a word called "BEGIN" and the last occurrence of a word called "END" using sed.

Input:

line1
BEGIN
line2 
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
line9
line10

Expected Output:

BEGIN
line2 
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END

My approach:

It extracts text between BEGIN and END. Here there are two BEGIN & END statement, and my solution extracts text between these words. My solution fails to extract text between first occurence of word1 (BEGIN) and last occurence of word2 (END).

dsonachalam$ sed -n -e '/^BEGIN$/,/^END$/p' logs.txt
BEGIN
line2 
line3
END
BEGIN
line6
line7
ENDED
END
sed
5 Answers
start=$(grep -n "BEGIN" $FILE_NAME |cut -f1 -d:|head -n 1)
end=$(grep -n "END" $FILE_NAME |cut -f1 -d:|tail -n 1)

sed -n $start,"$end"p $FILE_NAME
user_120070410

If the file is small enough to fit memory:

$ perl -0777 -ne 'print /(^BEGIN\n.*^END\n)/ms' ip.txt
BEGIN
line2 
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
user_40820520

With a 2-pass approach to avoid having to store any text in memory so it'll work for any size input file and with 1 call to 1 standard UNIX tool to avoid spawning multiple subshells, the following will work using any awk in any shell on every UNIX box:

$ awk '
    NR==FNR{ if (!beg && /BEGIN/) beg=NR; if (/END/) end=NR; next}
    (beg <= FNR) && (FNR <= end)
' file file
BEGIN
line2
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
user_17450010

A one-liner sed command would suffice (using GNU sed):

sed -E '/^BEGIN$/,$!d; :a; /(^|\n).*END$/{p;d}; $d; N; ba'

/^BEGIN$/,$!d; deletes lines above the first BEGIN. :a; /(^|\n).*END$/{p;d}; $d; N; ba accumulates ("slurps") lines into pattern space. Whenever an END line is read then the accumulated lines are printed out and pattern space is deleted starting a new cycle. Note that this "slurping" approach may be slow, or even may crash the sed process if the input is too large.

Content of input file:

line1
BEGIN
line2 
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END
line9
line10

and using GNU sed 4.8

sed -E '/^BEGIN$/,$!d; :a; /(^|\n).*END$/{p;d}; $d; N; ba' inputfile

prints

BEGIN
line2 
line3
END
line4
line5
BEGIN
line6
line7
ENDED
END
line8
END

Another approach would be:

lastend=$(sed -n '/^END$/=' inputfile | tail -1)
[[ -n $lastend ]] && sed -n "/^BEGIN\$/,${lastend}p" inputfile

This two-pass approach doesn't suffer from "slurping" lines.

user_138090010

This might work for you (GNU sed):

sed -n '/\<BEGIN\>/{x;:a;n;/\<END\>/{x;p;ba};H;$!ba;x;//P}' file

Set automatic printing off by using the -n option and then focus on lines following one that contains the word BEGIN.

Swap to the hold space (HS) and initiate a loop that fetches the next line and if that line contains the word END swap to the HS, print its contents and repeat.

If the current line does not contain the word END, append the current line to the HS and unless it is the end of file repeat.

At the end of file, print the first line of the HS if it begins END and whatever the condition allow the file processing to terminate.

Thus processing of lines only occurs once the word BEGIN has been seen and printing of those lines every time the word END occurs.

user_9674920
Related