How to trim whitespace from a Bash variable?

Viewed 1744985

I have a shell script with this code:

var=`hg st -R "$path"`
if [ -n "$var" ]; then
    echo $var
fi

But the conditional code always executes, because hg st always prints at least one newline character.

  • Is there a simple way to strip whitespace from $var (like trim() in PHP)?

or

  • Is there a standard way of dealing with this issue?

I could use sed or AWK, but I'd like to think there is a more elegant solution to this problem.

50 Answers

Bash has a feature called parameter expansion, which, among other things, allows string replacement based on so-called patterns (patterns resemble regular expressions, but there are fundamental differences and limitations). [flussence's original line: Bash has regular expressions, but they're well-hidden:]

The following demonstrates how to remove all white space (even from the interior) from a variable value.

$ var='abc def'
$ echo "$var"
abc def
# Note: flussence's original expression was "${var/ /}", which only replaced the *first* space char., wherever it appeared.
$ echo -n "${var//[[:space:]]/}"
abcdef

I've always done it with sed

  var=`hg st -R "$path" | sed -e 's/  *$//'`

If there is a more elegant solution, I hope somebody posts it.

You can delete newlines with tr:

var=`hg st -R "$path" | tr -d '\n'`
if [ -n $var ]; then
    echo $var
done

This is what I did and worked out perfect and so simple:

the_string="        test"
the_string=`echo $the_string`
echo "$the_string"

Output:

test

I've seen scripts just use variable assignment to do the job:

$ xyz=`echo -e 'foo \n bar'`
$ echo $xyz
foo bar

Whitespace is automatically coalesced and trimmed. One has to be careful of shell metacharacters (potential injection risk).

I would also recommend always double-quoting variable substitutions in shell conditionals:

if [ -n "$var" ]; then

since something like a -o or other content in the variable could amend your test arguments.

There are a few different options purely in BASH:

line=${line##+([[:space:]])}    # strip leading whitespace;  no quote expansion!
line=${line%%+([[:space:]])}   # strip trailing whitespace; no quote expansion!
line=${line//[[:space:]]/}   # strip all whitespace
line=${line//[[:space:]]/}   # strip all whitespace

line=${line//[[:blank:]]/}   # strip all blank space

The former two require extglob be set/enabled a priori:

shopt -s extglob  # bash only

NOTE: variable expansion inside quotation marks breaks the top two examples!

The pattern matching behaviour of POSIX bracket expressions are detailed here. If you are using a more modern/hackable shell such as Fish, there are built-in functions for string trimming.

I had to test the result (numeric) from a command but it seemed the variable with the result was containing spaces and some non printable characters. Therefore even after a "trim" the comparison was erroneous. I solved it by extracting the numerical part from the variable:

numerical_var=$(echo ${var_with_result_from_command} | grep -o "[0-9]*")

The simplest and cheapest way to do this is to take advantage of echo ignoring spaces. So, just use

dest=$(echo $source)

for instance:

> VAR="   Hello    World   "
> echo "x${VAR}x"
x   Hello    World   x
> TRIMD=$(echo $VAR)
> echo "x${TRIMD}x"
xHello Worldx

Note that this also collapses multiple whitespaces into a single one.

Array assignment expands its parameter splitting on the Internal Field Separator (space/tab/newline by default).

words=($var)
var="${words[@]}"

The "trim" function removes all horizontal whitespace:

ltrim () {
    if [[ $# -eq 0 ]]; then cat; else printf -- '%s\n' "$@"; fi | perl -pe 's/^\h+//g'
    return $?
}

rtrim () {
    if [[ $# -eq 0 ]]; then cat; else printf -- '%s\n' "$@"; fi | perl -pe 's/\h+$//g'
    return $?
}

trim () {
    ltrim "$@" | rtrim
    return $?
}

read already trims whitespace, so in bash you can do this:

$ read foo <<< "   foo  bar   two spaces follow   "
$ echo ".$foo."
.foo  bar   two spaces follow.

The POSIX compliant version is a bit longer

$ read foo << END
   foo  bar   two spaces follow   
END
$ echo ".$foo."
.foo  bar   two spaces follow.

The simplest way for the single-line use cases I know of:

echo "  ABC  " | sed -e 's# \+\(.\+\) \+#\1#'

How it works:

  • -e enables advanced regex
  • I use # with sed as I don't like the "messy library" patterns like /\////\/\\\/\/
  • sed wants most regex control chars escaped, hence all the \
  • Otherwise it's just ^ +(.+) +$, i.e. spaces at the beginning, a group no.1, and spaces at the end.
  • All this is replaced with just "group no.1".

Therefore, ABC becomes ABC.

This should be supported on most recent systems with sed.


For tabs, that would be

echo "  ABC  " | sed -e 's#[\t ]\+\(.\+\)[\t ]\+#\1#'

For multi-line content, that already needs character classes like [:space:] as described in other answers, and may not be supported by all sed implementations.

Reference: Sed manual

Create an array instead of variable this will trim all space, tab and newline characters:

arr=( $(hg st -R "$path") )
if [[ -n "${arr[@]}" ]]; then
    printf -- '%s\n' "${arr[@]}"
fi
Related