Replace/Remove lines between two other lines in Bash

Viewed 183

I have this file which is a Linux Device Tree:

# Other content
...

&uart3 {
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_uart3>;
    status = "okay";
};

# Other content
...

At first I wanted to extract lines between &uart3 and }; which is the UART3 node, with sed:

sed -n '/^&uart3/,${p;/^};/q}' uart3.dts

Output is:

&uart3 {
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_uart3>;
    status = "okay";
};

My problem, is how to remove all lines between &uart3 and }; , or is there a way to replace them with other content.

I read the awk solution about detecting a match and raising certain flag here. But I don't understand how to achieve this.

I'm not parsing the Device Tree here so no need for dtc library,

I'm handling the file as a text file only.

Since this gonna be ran into a Yocto recipe, the solution can be in Python as well.

4 Answers

In native bash:

retrieve_only_section() {
  local start end reading
  start=$1; end=$2; reading=0
  while IFS= read -r line; do
    if [[ $line = $start ]]; then
      reading=1
    elif [[ $line = $end ]]; then
      reading=0
      printf '%s\n' "$line"
      continue
    fi
    (( reading )) && printf '%s\n' "$line"
  done
}

...and retrieve_only_section '&uart3 {' '};' <yourfile will retrieve only the section.

Similarly, one can just replace (( reading )) && printf '%s\n' "$line" with (( reading )) || printf '%s\n' "$line" to print only things outside the section.

To replace the section, one might use:

replace_section() {
  local start end replacement in_section
  start=$1; end=$2; replacement=$3; in_section=0
  while IFS= read -r line; do
    if (( ! in_section )) && [[ $line = $start ]]; then
      in_section=1
    elif (( in_section )) && [[ $line = $end ]]; then
      in_section=0
      printf '%s\0' "$replacement"
      continue
    fi
    (( in_section )) || printf '%s\n' "$line"
  done
}

This is a variant of what seems the most frequent question I answer, lol

$: cat file
# Other content
...

&uart3 {
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_uart3>;
    status = "okay";
};

# Other content
...

$: cat replacement.txt
   This is what I want there instead.
   I just stuck it over here in this other file.

$: cat tst
sed '/^&uart3 {/,/^};/{
  /^&uart3 {/{ p; r replacement.txt
             }
  /^};/!d;
}' file

$: ./tst
# Other content
...

&uart3 {
   This is what I want there instead.
   I just stuck it over here in this other file.
};

# Other content
...

/^&uart3 {/,/^};/{ opens a block on any line in the matched range from ^&uart3 { to the next ^};
The next bit has a gotcha - don't put anything on the same line after a filename.

      /^&uart3 {/{ p; r replacement.txt
                 }

That says if the line we're looking at is the ^&uart3 { then open a block -
first print it, then read in the contents of replacement.txt.
You have to close the block on the next line in most implementations, though, so it doesn't think the curly is part of the file name and break the syntax.

/^};/!d; says for the rest of the block, unless (!) it's the closing curly, just delete it.


#edit

If you want to do it without a file -

$: cat tst
sed '/^&uart3 {/,/^};/{
  /^&uart3 {/{ p; a\
   This is what I want there instead.\
   There is no other file.
             }
  /^};/!d;
}' file

$: ./tst
# Other content
...

&uart3 {
   This is what I want there instead.
   There is no other file.
};

# Other content
...

c.f. https://www.gnu.org/software/sed/manual/sed.html#Other-Commands
It's all there. :)

If ed is available/acceptable.

printf '%s\n' '/^&uart3 {$/+1;/^};$/-1c'  foo .  ,p Q | ed -s file.txt

Output

# Other content
...

&uart3 {
foo
};

# Other content
...

Or use an array to store the new data/input.

#!/usr/bin/env bash

input_data=(
'  foo'
'  bar'
'  baz'
'  more'
'  qux'
)

printf '%s\n' '/^&uart3 {$/+1;/^};$/-1c'  "${input_data[@]}" .  ,p Q | ed -s file.txt

Output

# Other content
...

&uart3 {
  foo
  bar
  baz
  more
  qux
};

# Other content
...

Or use a separate file for the new input/data

input_file.txt

the
quick
brown
fox
jumps
over
the
lazy
dog

Then

printf '%s\n' '/^&uart3/+1;/^};/-1d' '-1r input_file.txt' ,p Q | ed -s file.txt

Output

# Other content
...

&uart3 {
the
quick
brown
fox
jumps
over
the
lazy
dog
};

# Other content
...

  • Change Q to w if you're ok with the output and in-place editing of the file.txt will occur.

  • Remove the ,p to silence the output.

A) remove the lines with sed

sed '5,7d' file

5,7 is a range [lines to delete] this are inclusive, meaning is going to delete line 5,6,7. d argument is to delete in sed. Taking your example this is going to return the following:

&uart3 {
};

B) replace ONE line with sed

sed '5s/[a-z].*/stringToReplace/' file  

5 selects the number line[5] the s for replace string, [a-z] is for selecting any letter in lowercase, in this case: pinctrl. .* is for selecting [the whole line]. output:

&uart3 {
  stringToReplace
  pinctrl-0 = <&pinctrl_uart3>;
  status = "okay";
};

C) replace all lines with sed from another text file

sed '5,+2d;4r replace.txt' file  

if you have another file with TEXT then use r ARGUMENT replace.txt. 5,+2d selects the line number 5. +2 selects two lines after 5, meaning, 5,6,7.d deletes the lines. ; is for adding another sed command[like pipes]. 4r means, selects line number 4. r means replace with a text file, in this case replace.txt. output

&uart3 {
  replace = line 5
  replace = line 6
  replace = line 7
};

You need to know the number of lines to be removed:

grep -n "pinctrl-\|status" file  

-n returns the number of lines. \| this is for multiples arguments with grep. 'example1\|example2\|exmple3' --> returns:

enter image description here

Related