How to remove or replace a multiline text between two patterns

user_10734452 0
Viewed 118

I would like to add some customer flags in some of my scripts for it to be parsed before packaging by a shell script.

Let's say, removing all multiline text between

^([#]|[//]){0,1}[_]+NOT_FOR_CUSTOMER_BEGIN[_]+\n

and between

^([#]|[//]){0,1}[_]+NOT_FOR_CUSTOMER_END[_]+\n

I want it to be error tolerant (about the number of '_') that is why i'm using regexes.

For instance:

before.foo

i want this
#____NOT_FOR_CUSTOMER_BEGIN________
not this
nor this
#________NOT_FOR_CUSTOMER_END____
and this
//____NOT_FOR_CUSTOMER_BEGIN__
not this again
nor this again
//__________NOT_FOR_CUSTOMER_END____
and this again

will become:

after.foo

i want this
and this
and this again

I'd rather use sed but any clever solution will be welcomed :)

Something like this:

cat before.foo |  tr '\n' '\a' | sed -r 's/([#]|[//]){0,1}[_]+NOT_FOR_CUSTOMER_BEGIN[_]+\a.*\a([#]|[//]){0,1}[_]+NOT_FOR_CUSTOMER_END[_]+\a/\a/g' | tr '\a' '\n' > after.foo
sed substitution awk shell python
4 Answers

sed is easiest tool to handle this as it can delete lines from a start pattern to end patter:

sed -E '/_+NOT_FOR_CUSTOMER_BEGIN_+/,/_+NOT_FOR_CUSTOMER_END_+/d' file

i want this
and this
and this again

If you are looking for awk solution then here is a simpler awk:

awk '/_+NOT_FOR_CUSTOMER_BEGIN_+/,/_+NOT_FOR_CUSTOMER_END_+/{next} 1' file
user_5482250

Have awk solution in this way, written and tested with your shown samples.

awk '
/^([#]|[/][/])__+NOT_FOR_CUSTOMER_BEGIN/{ found=1       }
/^([#]|[/][/])__+NOT_FOR_CUSTOMER_END/  { found=""; next}
!found
'  Input_file

With your shown samples, output will be as follows.

i want this
and this
and this again

Explanation: Simple explanation is: whenever start string(with regex) is found make a flag to TRUE(for non printing) and whenever end string(with regex check) comes Nullify the flag to start printing(depending upon the lines) next line onwards.

user_58665800

You could use a Python script:

import re

data = """
i want this
#____NOT_FOR_CUSTOMER_BEGIN________
not this
nor this
#________NOT_FOR_CUSTOMER_END____
and this
//____NOT_FOR_CUSTOMER_BEGIN__
not this again
nor this again
//__________NOT_FOR_CUSTOMER_END____
and this again
"""

rx = re.compile(r'^(#|//)(?:.+\n)+^\1.+\n?', re.MULTILINE)
data = rx.sub('', data)
print(data)

Which would yield

i want this
and this
and this again

See a demo on regex101.com.

user_12314500

You could match as least as possible lines from NOT_FOR_CUSTOMER_BEGIN_ till NOT_FOR_CUSTOMER_END_

Note that [//] matches a single / instead of //

^(?:#|//)_+NOT_FOR_CUSTOMER_BEGIN_+(?:\n.*)*?\n(?:#|//)_+NOT_FOR_CUSTOMER_END_+\n*
  • ^ Start of string
  • (?:#|//) Match either # or //
  • _+NOT_FOR_CUSTOMER_BEGIN_+ Match NOT_FOR_CUSTOMER_BEGIN between 1 or more underscores
  • (?:\n.*)*? Repeat as least as possible lines
  • \n(?:#|//)_+NOT_FOR_CUSTOMER_END_+ Match a newline, then either # or // and NOT_FOR_CUSTOMER_END_ between one or more underscores
  • \n* Remove optional trailing newlines

Regex demo

Another way to use it with Python:

import re

regex = r"^(?:#|//)_+NOT_FOR_CUSTOMER_BEGIN_+(?:\n.+)*?\n(?:#|//)_+NOT_FOR_CUSTOMER_END_+\n*"

s = ("i want this\n"
            "#____NOT_FOR_CUSTOMER_BEGIN________\n"
            "not this\n"
            "nor this\n"
            "#________NOT_FOR_CUSTOMER_END____\n"
            "and this\n"
            "//____NOT_FOR_CUSTOMER_BEGIN__\n"
            "not this again\n"
            "nor this again\n"
            "//__________NOT_FOR_CUSTOMER_END____\n"
            "and this again")

subst = ""
result = re.sub(regex, "", s, 0, re.MULTILINE)

if result:
    print (result)

Output

i want this
and this
and this again
user_54249880
Related