How can I replace each newline (\n) with a space using sed?

Viewed 2074123

How can I replace a newline ("\n") with a space ("") using the sed command?

I unsuccessfully tried:

sed 's#\n# #g' file
sed 's#^$# #g' file

How do I fix it?

43 Answers

sed is intended to be used on line-based input. Although it can do what you need.


A better option here is to use the tr command as follows:

tr '\n' ' ' < input_filename

or remove the newline characters entirely:

tr -d '\n' < input.txt > output.txt

or if you have the GNU version (with its long options)

tr --delete '\n' < input.txt > output.txt

Use this solution with GNU sed:

sed ':a;N;$!ba;s/\n/ /g' file

This will read the whole file in a loop (':a;N;$!ba), then replaces the newline(s) with a space (s/\n/ /g). Additional substitutions can be simply appended if needed.

Explanation:

  1. sed starts by reading the first line excluding the newline into the pattern space.
  2. Create a label via :a.
  3. Append a newline and next line to the pattern space via N.
  4. If we are before the last line, branch to the created label $!ba ($! means not to do it on the last line. This is necessary to avoid executing N again, which would terminate the script if there is no more input!).
  5. Finally the substitution replaces every newline with a space on the pattern space (which is the whole file).

Here is cross-platform compatible syntax which works with BSD and OS X's sed (as per @Benjie comment):

sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g' file

As you can see, using sed for this otherwise simple problem is problematic. For a simpler and adequate solution see this answer.

The Perl version works the way you expected.

perl -i -p -e 's/\n//' file

As pointed out in the comments, it's worth noting that this edits in place. -i.bak will give you a backup of the original file before the replacement in case your regular expression isn't as smart as you thought.

In order to replace all newlines with spaces using awk, without reading the whole file into memory:

awk '{printf "%s ", $0}' inputfile

If you want a final newline:

awk '{printf "%s ", $0} END {printf "\n"}' inputfile

You can use a character other than space:

awk '{printf "%s|", $0} END {printf "\n"}' inputfile

Why didn't I find a simple solution with awk?

awk '{printf $0}' file

printf will print the every line without newlines, if you want to separate the original lines with a space or other:

awk '{printf $0 " "}' file

If you are unfortunate enough to have to deal with Windows line endings, you need to remove the \r and the \n:

tr '\r\n' ' ' < $input > $output

I'm not an expert, but I guess in sed you'd first need to append the next line into the pattern space, bij using "N". From the section "Multiline Pattern Space" in "Advanced sed Commands" of the book sed & awk (Dale Dougherty and Arnold Robbins; O'Reilly 1997; page 107 in the preview):

The multiline Next (N) command creates a multiline pattern space by reading a new line of input and appending it to the contents of the pattern space. The original contents of pattern space and the new input line are separated by a newline. The embedded newline character can be matched in patterns by the escape sequence "\n". In a multiline pattern space, the metacharacter "^" matches the very first character of the pattern space, and not the character(s) following any embedded newline(s). Similarly, "$" matches only the final newline in the pattern space, and not any embedded newline(s). After the Next command is executed, control is then passed to subsequent commands in the script.

From man sed:

[2addr]N

Append the next line of input to the pattern space, using an embedded newline character to separate the appended material from the original contents. Note that the current line number changes.

I've used this to search (multiple) badly formatted log files, in which the search string may be found on an "orphaned" next line.

cat file | xargs

for the sake of completeness

I used a hybrid approach to get around the newline thing by using tr to replace newlines with tabs, then replacing tabs with whatever I want. In this case, "
" since I'm trying to generate HTML breaks.

echo -e "a\nb\nc\n" |tr '\n' '\t' | sed 's/\t/ <br> /g'`

Finds and replaces using allowing \n

sed -ie -z 's/Marker\n/# Marker Comment\nMarker\n/g' myfile.txt

Marker

Becomes

# Marker Comment

Marker

sed '1h;1!H;$!d
     x;s/\n/ /g' YourFile

This does not work for huge files (buffer limit), but it is very efficient if there is enough memory to hold the file. (Correction H-> 1h;1!H after the good remark of @hilojack )

Another version that change new line while reading (more cpu, less memory)

 sed ':loop
 $! N
 s/\n/ /
 t loop' YourFile

This might work for you (GNU sed):

sed 'H;$!d;x;:a;s/^((.).*)\2/\1 /;ta;s/.//' file

The H command prepends a newline to the pattern space and then appends the result to the hold space. The normal flow of sed is to remove the following newline from each line, thus this will introduce a newline to the start of the hold space and the replicate the remainder of the file. Once the file has been slurped into the hold space, swap the hold space with the patten space and then use pattern matching to replace all original newlines with spaces. Finally, remove the introduced newline.

This has the advantage of never actually entering a newline string within the sed commands.

Alternative:

sed 'H;$!d;x;y/\n/ /;s/.//' file

Or:

sed 'H;1h;$!d;x;y/\n/ /' file

Yet another option:

tr -s "[:space:]" " " < filename > filename2 && mv filename2 filename

Where tr -s is for:

-s, --squeeze-repeats replace each sequence of a repeated character that is listed in the last specified SET, with a single occurrence of that character

This uses replaces all whitespace sequences in the file with a single space, writes result to a new file, then renames new file back to original name.

@OP, if you want to replace newlines in a file, you can just use dos2unix (or unix2dox)

dos2unix yourfile yourfile

Try this:

echo "a,b"|sed 's/,/\r\n/'
Related