Using sed to replace unique text with URL

Question

I'm trying to figure out a more elegant way to replace a unique piece of text in a file with a URL.

It seems that sed is interpreting the URL as part of its evaluation logic instead of just replacing the text from a bash variable.

My file looks something like:

$srcRemoveSoftwareURL = "softwareURL"

and I'm attempting to (case-sensitive) search/replace softwareURL with the actual URL.

I'm using a bash script to help with the manipulation and I'm setting up my variables like so:

STORAGE_ENDPOINT_URL="http://mywebsite.com"

sas_url="se=2021-07-20T18%3A42Z&sp=rl&spr=https&sv=2018-11-09&sr=s&sig=oI/T9oHqzfEtuTjAotLyLN3IXbkiADGTPQllkyJlvEA%3D"

softwareURL="$STORAGE_ENDPOINT_URL/1-remove-software.sh?$sas_url"

# the resulting URL is like this:
# http://mywebsite.com/1-remove-software.sh?se=2021-07-20T18%3A42Z&sp=rl&spr=https&sv=2018-11-09&sr=s&sig=oI/T9oHqzfEtuTjAotLyLN3IXbkiADGTPQllkyJlvEA%3D

I then use sed to replace the text:

sed "s|softwareURL|$softwareURL|" template_file.sh

I recognize that bash is taking preference for the $softwareURL variable and inserting it in, but then sed interprets the URL as part of some evaluation logic.

At the moment my resulting template file looks like so:

$srcRemoveSoftwareURL = "http://mywebsite.com/1-remove-software.sh?se=2021-07-20T18%3A42ZsoftwareURLsp=rlsoftwareURLspr=httpssoftwareURLsv=2018-11-09softwareURLsr=ssoftwareURLsig=oI/T9oHqzfEtuTjAotLyLN3IXbkiIIGTPQllkyJlvEA%3D

It seems that sed is also finding any ampersand & characters in the URL and replacing it with the literal softwareURL.

What I'm doing now is to pipe the result to sed again and replace softwareURL with & which seems a little inefficient.

Is there a better way to do this?

Any guidance is most welcome!

Thanks!

user_7366100 · Answer

The issue with the current result is that in sed the & refers to the pattern matched in the first part of the sed/s command (in this case & == softwareURL).

One idea would be to escape all &'s in the replacement string, eg:

sed "s|softwareURL|$softwareURL|"          template_file.sh      # old
sed "s|softwareURL|${softwareURL//&/\&}|" template_file.sh      # new

With this new command generating:

$srcRemoveSoftwareURL = "http://mywebsite.com/1-remove-software.sh?se=2021-07-20T18%3A42Z&sp=rl&spr=https&sv=2018-11-09&sr=s&sig=oI/T9oHqzfEtuTjAotLyLN3IXbkiADGTPQllkyJlvEA%3D"

If you find yourself needing to make several replacements:

$ somevariable="abc & % $ 123 & % $ xyz"
$ somevariable="${somevariable//&/\&}"     # escape literal '&'
$ somevariable="${somevariable//%/\%}"     # escape literal '%'
$ somevariable="${somevariable//$/\$}"     # escape literal '$'
$ echo $somevariable
abc \& \% \$ 123 \& \% \$ xyz

NOTE: I'm not saying you need to escape these particular characters for sed ... just pointing out how to go about making multiple replacements in a variable.

Using sed to replace unique text with URL

1 Answers