How to convert only certain columns of whitespace into tab?

Viewed 143

I know I can use sed 's/[[:blank:]]/,/g' to convert blank spaces into commas or anything of my choosing in my file, but is there a way to somehow set it so that, only the first 5 instances of whitespace convert them into a comma?

This is because my last column has a lot of information written out, so it is annoying when sed coverts all the spaces in that column into commas.

Sample input file:

sample1 gi|11| 123 33 97.23 This is a sentence
sample2 gi|22| 234    33 97.05   Keep these spaces

And the output I was looking for:

sample1,gi|11|,123,33,97.23,This is a sentence
sample2,gi|22|,234,33,97.05,Keep these spaces

Only the first 5 chains of whitespace are changed to a comma.

5 Answers

With GNU awk for the 3rd arg to match():

$ awk '{ match($0,/((\S+\s+){5})(.*)/,a); gsub(/\s+/,",",a[1]); print a[1] a[3] }' file
sample1,gi|11|,123,33,97.23,This is a sentence
sample2,gi|22|,234,33,97.05,This is a sentence

but I'd recommend you actually turn it into a valid CSV (i.e. one that conforms to RFC 4180) such as could be read by MS-Excel and other tools since "This is a sentence" (and possibly other fields) can presumably include commas and double quotes:

$ awk '{
    gsub(/"/,"\"\"");
    match($0,/((\S+\s+){5})(.*)/,a)
    gsub(/\s+/,"\",\"",a[1])
    print "\"" a[1] a[3] "\""
}' file
"sample1","gi|11|","123","33","97.23","This is a sentence"
"sample2","gi|22|","234","33","97.05","This is a sentence"

For example given this input:

$ cat file
sample1 gi|11| 123 33 97.23 This is a sentence
a,b,sample2 gi|22| 234 33 97.05 This is, "typically", a sentence

The output from the first script is not valid CSV:

$ awk '{ match($0,/((\S+\s+){5})(.*)/,a); gsub(/\s+/,",",a[1]); print a[1] a[3] }' file
sample1,gi|11|,123,33,97.23,This is a sentence
a,b,sample2,gi|22|,234,33,97.05,This is, "typically", a sentence

while the output from the 2nd script IS valid CSV:

$ awk '{ gsub(/"/,"\"\""); match($0,/((\S+\s+){5})(.*)/,a); gsub(/\s+/,"\",\"",a[1]); print "\"" a[1] a[3] "\"" }' file
"sample1","gi|11|","123","33","97.23","This is a sentence"
"a,b,sample2","gi|22|","234","33","97.05","This is, ""typically"", a sentence"

perl's split can limit the number of fields with its third argument:

$ perl -lnE 'say join ",", split(" ",$_,6)' file
sample1,gi|11|,123,33,97.23,This is a sentence
sample2,gi|22|,234,33,97.05,This is a sentence

If fields might require quoting:

perl -lnE 'say join ",", map { s/"/""/g || y/,// ? "\"$_\"" : $_ } split(" ",$_,6)
' file

Ruby has a str.split that can take a limit:

ruby -ne 'puts $_.split(/\s+/,6).join(",")' file
sample1,gi|11|,123,33,97.23,This is a sentence
sample2,gi|22|,234,33,97.05,This is a sentence

As does Perl:

perl -lnE 'say join ",", split /\s+/,$_,6 ' file
# same

This might work for you (GNU sed):

sed -E 's/\s+/&\n/5;h;s/\n.*//;s/\s+/,/g;G;s/\n.*\n//' file

Append a newline to the 5th occurrence of a group of whitespace.

Make a copy of the amended line in the hold space.

Remove the section from the inserted newline to the end of the line.

Translate groups of whitespace into commas.

Append the copy.

Remove the section between newlines.

Thus the first five groups of whites space are converted to commas and the remaining groups are untouched.

Here is a way to do with 3 sed commands. This requires GNU sed, which supports the /ng (e.g. /6g) pattern flag. That will only apply the substitution from the nth occurrence on. Also note: this method will compress multiple spaces in the last column permanently.

sed 's/ \+/␣/6g' | sed 's/ \+/,/g' | sed 's/␣/ /g'

Another variation: Do the multiple space compression as a separate step with tr -s ' '. This may be more readable.

tr -s ' ' | sed 's/ /␣/6g' | sed 's/ /,/g' | sed 's/␣/ /g'

Another variation: Compress all whitespaces, not just spaces with \s

sed 's/\s\+/␣/6g' | sed 's/\s\+/,/g' | sed 's/␣/ /g'

Explanation:

  1. The first step "protects" the spaces from the 6th occurrence on, by converting them to a special character. I've used here (unicode U+2423), but you could use any character that doesn't exist in the source data, such as \x00, {space}, etc.
sed 's/ \+/␣/6g'
  1. The second step converts the remaining spaces to commas.
sed 's/ \+/,/g'
  1. The third step converts the "protected" spaces back to spaces.
sed 's/␣/ /g'
Related