Strange behavior of simple sed global replacement in different OSs

user_3387716 0
Viewed 75

I stumbled upon a little problem with the following sed command that strips the leading and trailing space characters of a line and encapsulates it in double-quotes:

printf '%s\n' ' hello ' ' hello' 'hello ' hello  | sed -E 's/^ *| *$/"/g'

The results are:

  • on Linux:
"hello"
"hello"
"hello"
"hello"
  • on macOS:
"hello"
"hello
"hello"
"hello
  • on FreeBSD
"hello"
"
"hello"
"hello"

I'm not really looking for a workaround because I got these ones that work in all platforms (I'm open to other proposals though):

sed 's/^ */"/;s/ *$/"/'
awk '{gsub(/^ *| *$/,"\"")}1' # the culprit works fine with awk

My question is: Is my understanding of the sed command wrong or can this be considered a bug in macOS and FreeBSD sed implementations?

sed
1 Answers

Sed Implementations Vary Widely

There are so many different versions of sed that in general you simply shouldn't expect standardized behavior between different versions, especially on different operating systems. The sed FAQ is arguably out of date, but gives a good sense of how many different implementations there are. The Open Group also provides a base specification for sed, which is primarily notable for the fact that the -E flag and support for extended regular expressions (ERE) are not included.

GNU sed is arguably more portable in the sense that it can be compiled and will behave similarly on any system where it can be built with GCC, any desired extensions such as ERE and PCRE support, and where line endings are either the same or are accounted for properly in your expressions. Since most sed implementations are really supersets of the POSIX specifications, you are otherwise at the mercy of the particular sed you are using for:

  1. actual POSIX compliance, as not all shells, OSes, or userland tools aim for full POSIX compliance with or without flags; and
  2. any additional features you may need for your particular use case or expressions.

If portability is a primary concern, and you can't rely on having GNU sed, then you might want to see if you can rely on other sed-like supersets like the Perl or Ruby modes that imitate sed using command-line flags for line-oriented processing but with much more powerful regular expression engines under the hood.

user_13019720
Related