How to delete certain characters after a pattern using sed or awk?

user_14439183 0
Viewed 95

I have a text file containing number of lines formatted like below

001_A.wav;112.680;115.211;;;Ja. Hello; Hi:

my goal is to clean whatever is after ;;;. Meaning to delete the following characters ,;()~?

I know i can do something like sed 's/[,.;()~?,]//g'. However if I do that, it would give me something like

001_Awav112.680115211Ja Hello Hi

However I would like to delete those character only after ;;; so I would get

001_A.wav;112.680;115.211;;;Ja Hello Hi

How can I accomplish this task?

sed awk shell bash
5 Answers

1st solution: Could you please try following, written and tested with shown samples in GNU awk(where assuming ;;; occurring one time in lines).

awk '
match($0,/.*;;;/){
  laterPart=substr($0,RSTART+RLENGTH)
  gsub(/[,.:;()~?]/,"",laterPart)
  print substr($0,RSTART,RLENGTH) laterPart
}'  Input_file

Explanation: Adding detailed explanation for above.

awk '                                  ##Starting awk program from here.
match($0,/.*;;;/){                     ##Using atch function to match everything till ;;; here.
  laterPart=substr($0,RSTART+RLENGTH)  ##Creating variable laterPart which has rest of the line apart from matched regex part above.
  gsub(/[,.:;()~?]/,"",laterPart)      ##Globally substituting ,.:;()~? with NULL in laterPart variable.
  print substr($0,RSTART,RLENGTH) laterPart ##Printing sub string of matched regex and laterPart var here.
}' Input_file                          ##Mentioning Input_file name here.

2nd solution: In case you have multiple occurrences of ;;; in lines and you want to substitute characters from all fields, after 1st occurrence of ;;; then try following.

awk 'BEGIN{FS=OFS=";;;"} {for(i=2;i<=NF;i++){gsub(/[,.:;()~?,]/,"",$i)}} 1' Input_file
user_58665800

You can use

sed ':a; s/\(;;;[^,.:;()~?,]*\)[,.:;()~?,]/\1/; ta' file > newfile
sed ':a; s/\(;;;[^[:punct:]]*\)[[:punct:]]/\1/; ta' file > newfile

Details

  • :a sets a label
  • \(;;;[^,.:;()~?,]*\)[,.:;()~?,] matches and captures into Group 1 a ;;; substring and then any zero or more chars other than ,.:;()~?, chars, and then just matches a char from the ,.:;()~?, set
  • [^[:punct:]]* matches any 0 or more chars other than punctuation chars
  • [[:punct:]] matches any punctuation char
  • \1 is the replacement, the contents of Group 1
  • ta branches back to a label on a successful replacement.

See the online sed demo:

s='001_A.wav;112.680;115.211;;;Ja. Hello; Hi:'
sed ':a; s/\(;;;[^,.:;()~?,]*\)[,.:;()~?,]/\1/; ta' <<< "$s"
# => 001_A.wav;112.680;115.211;;;Ja Hello Hi
sed ':a; s/\(;;;[^[:punct:]]*\)[[:punct:]]/\1/; ta' <<< "$s"
# => 001_A.wav;112.680;115.211;;;Ja Hello Hi
user_38329700

Didn't read your question correctly, but I've changed it now.

I suggest to make use of perl instead, since it has lookup groups.

$ perl -pe 's/^((?:(?!;;;).)*;;;)|[:,.;\(\)~\?,]/\1/g' file.txt
  • ^ is the beginning of the line.
  • ((?:(?!;;;).)*;;;) is the string equivalent of [^;]*, and makes sure that the first ;;; is found and groups it in \1.
  • |[:,\.;\(\)~\?,] selects the characters :,.;\(\)~\?, and denies it in the result. (Thus leaving "Ja" in it).
user_83829290

You can use the combination of some sed commands with

echo '001_A.wav;112.680;115.211;;;Ja. Hello; Hi:' |
   sed 's/;;;/;;;\n\r/' |
   sed  '/^\r/ s/[,;():~?]//g' |
   sed -z 's/;;;\n\r/;;;/g'
user_32201130

Different GNU AWK-solution:

echo "001_A.wav;112.680;115.211;;;Ja. Hello; Hi:" | awk 'BEGIN{FS=OFS=";;;"}{print $1,gensub(/[,;()~?]/,"","g",substr($0,length($1)+1))}'

output:

001_A.wav;112.680;115.211;;;Ja. Hello Hi:

This assumes your description has precedence over example (only ,;()~? will be removed). Explanation: I use ;;; as seperator and output seperator then I print 1st column (what is before ;;;) and get rest by finding its start as length of 1st column plus 1, then remove all specified characters from that part and print it. If example has precedence over description then you might use [[:punct:]] set of characters, namely:

echo "001_A.wav;112.680;115.211;;;Ja. Hello; Hi:" | awk 'BEGIN{FS=OFS=";;;"}{print $1,gensub(/[[:punct:]]/,"","g",substr($0,length($1)+1))}'

will give

001_A.wav;112.680;115.211;;;Ja Hello Hi
user_107859750
Related