How can I do a recursive find/replace of a string with awk or sed?

Viewed 685937

How do I find and replace every occurrence of:

subdomainA.example.com

with

subdomainB.example.com

in every text file under the /home/www/ directory tree recursively?

37 Answers

A straight forward method if you need to exclude directories (--exclude-dir=..folder) and also might have file names with spaces (solved by using 0Byte for both grep -Z and xargs -0)

grep -rlZ oldtext . --exclude-dir=.folder | xargs -0 sed -i 's/oldtext/newtext/g'

Simplest way to replace (all files, directory, recursive)

find . -type f -not -path '*/\.*' -exec sed -i 's/foo/bar/g' {} +

Note: Sometimes you might need to ignore some hidden files i.e. .git, you can use above command.

If you want to include hidden files use,

find . -type f  -exec sed -i 's/foo/bar/g' {} +

In both case the string foo will be replaced with new string bar

or use the blazing fast GNU Parallel:

grep -rl oldtext . | parallel sed -i 's/oldtext/newtext/g' {}
#!/usr/local/bin/bash -x

find * /home/www -type f | while read files
do

sedtest=$(sed -n '/^/,/$/p' "${files}" | sed -n '/subdomainA/p')

    if [ "${sedtest}" ]
    then
    sed s'/subdomainA/subdomainB/'g "${files}" > "${files}".tmp
    mv "${files}".tmp "${files}"
    fi

done

According to this blog post:

find . -type f | xargs perl -pi -e 's/oldtext/newtext/g;'

Here's a version that should be more general than most; it doesn't require find (using du instead), for instance. It does require xargs, which are only found in some versions of Plan 9 (like 9front).

 du -a | awk -F' '  '{ print $2 }' | xargs sed -i -e 's/subdomainA\.example\.com/subdomainB.example.com/g'

If you want to add filters like file extensions use grep:

 du -a | grep "\.scala$" | awk -F' '  '{ print $2 }' | xargs sed -i -e 's/subdomainA\.example\.com/subdomainB.example.com/g'

Replacing find(1) with the simpler fd(1)/fdfind = https://github.com/sharkdp/fd:

fdfind . --type f --exec sed -i "s/original_string/new_string/g"

Addressing fd(1) iconsistent pkg & cmd names

  • on macOS homebrew: pkg and cmd = fd
  • on Ubuntu 20.04: pkg = fd-find, cmd = fdfind

I make an alias fdfind='fd' on macOS for consistent cmd naming (between my macOS and Linux platforms).

More on this point at https://github.com/sharkdp/fd/issues/1009.

More details and additional features

# bash examples:

1='original_string'
2='new______string'

# for this (the original-poster's) question:
1='subdomainA.example.com'
2='subdomainB.example.com'

# 'fdfind' (on at least Ubuntu 20.04) = 'fd' = https://github.com/sharkdp/fd

fdfind . --type f --exec sed -i "s/$1/$2/g"

# Here's a slightly-more-complex example that
# a. excludes (-E) .git/ and archive/ dirs, and
# b. performs a word-boundary search on the original_string (\<$1\>):
fdfind . -E .git/ -E archive/ --type f --exec sed -i "s/\<$1\>/$2/g"

Even fancier: controlling the word-boundary-ness from the third ($3) command-line paramter (third parameter = noword means no boundary, leftword means only left-side word boundary, rightword means only right-side boundary):

#!/usr/bin/env bash

#
# replace-tree.bash
#

# 'fdfind' (on at least Ubuntu 20.04) = 'fd' = https://github.com/sharkdp/fd

if [ $# -lt 2 ]; then
  echo "$0: Please provide at least 2 arguments."
  exit 1
fi

original="\<$1\>"

if   [ "$3" = "noword" ];    then
  original="$1"
elif [ "$3" = "leftword" ];  then
  original="\<$1"
elif [ "$3" = "rightword" ]; then
  original="$1\>"
fi

fdfind . --type f --exec sed -i "s/$original/$2/g"

Example usage:

$ replace-tree.bash original_string new_string leftword
$

I'm surprised I've not seen the simple answer using file globbing, which I just used to scan/update ONLY packge.json files with **/package.json

this was macos specific under zsh

cd /home/www
sed -i '' -e 's/subdomainA.example.com/subdomainA.example.com/g' **/*
Related