Perl one-liner Windows 10 / Power Shell : replace a string in multiple file ending by *.sql won't work (* globing not recognized)

Viewed 245

The following one-liner Perl statement didn't work when using the "*" globbing:

>>> perl -i'.bak' -pe 's#__SUB__CWD__#'$(pwd)'#g' *.sql
Can't open *.sql: Invalid argument. 

The following statement works but only when you give a single file as an argument:

>>> perl -p -i'.bak' -e 's#__SUB__CWD__#'$(pwd)'#g' A_SINGLE_FILE.sql

Any idea what could make perl to realize "*" must be read in a REGEX way?

I'm trying to replace all the occurence of "SUB__CWD" in tons of files.

C.f. https://github.com/oracle/db-sample-schemas/blob/master/README.md

2 Answers

That's not Perl's fault, it's Windows' fault. Linux shells expand the wildcards for you, cmd.exe does not. I don't know about PowerShell, but it seems it doesn't either.

Also, you don't want to interpret the asterisk the regex way, but the glob pattern way. In a regex, * means "the previous thing should be repeated zero or more times".

You can use Perl's glob function:

perl -p -i'.bak' -e 'BEGIN {@ARGV = glob shift} s#__SUB__CWD__#'$(pwd)'#g' *.sql

Note that you need to backslash backslashes in the path to prevent Perl from interpreting them, i.e. use $($pwd -replace '\\', '\\') instead of just $(pwd). You also need to backslash the # characters in the path as they would otherwise be understood as the delimiters, so use $($pwd -replace '[\\#]', '\$&').

Therefore, it would be even cleaner not to use $(pwd), but let Perl know what the replacement is:

perl -p -i'.bak' -e 'BEGIN{ ($pwd, @ARGV) = (shift, map glob, @ARGV) }
                   s/__SUB__CWD__/$pwd/g' -- $(pwd) *.sql

The first argument is assigned to $pwd via shift, the remaining attributes are mapped to glob, which means you can specify more than one wildcard pattern.

choroba's answer provides good pointers and solutions focused more on the perl side of things.

A direct adaptation of your command to PowerShell on Windows is (both Windows PowerShell and PowerShell [Core]):

perl -i'.bak' -pe "s#__SUB__CWD__#$($pwd -replace '[\\#]', '\$&')#g" (Get-ChildItem *.sql).Name

As choroba points out, PowerShell on Windows does not do globbing (expansion of wildcard patterns to matching filenames) for you; however, it does on Unix-like platforms - see below.

  • (Get-ChildItem *.sql).Name uses the Get-ChildItem cmdlet to perform the globbing manually.

  • $pwd -replace '[\\#]', '\$&' uses PowerShell's regex-based -replace operator to ensure that both the \ path separators in the current-directory path as well as any # characters are \-escaped, to ensure that \ isn't interpreted as the start of an escape sequence by Perl s function, and that a # isn't misinterpreted as a delimiter character.

For PowerShell [Core] on Unix-like platforms (macOS, Linux):

Essentially, your command would work as-is - as long as the current directory path doesn't contain spaces (which would affect your command in POSIX-like shells such as bash too).

The following variant addresses this problem:

# PowerShell [Core] on macOS and Linux
# If the current-directory path contains '#' chars.,
# use the -replace operation shown above.
perl -i'.bak' -pe "s#__SUB__CWD__#$(pwd)#g" *.sql

Note: $($pwd) would be slightly more efficient, because it uses the automatic $PWD variable rather than the pwd command, an alias for the Get-Location cmdlet.

Related