Regular expression - replace all spaces in beginning of line with periods

Viewed 1506

I don't care if I achieve this through vim, sed, awk, python etc. I tried in all, could not get it done.

For an input like this:

top           f1    f2    f3
   sub1       f1    f2    f3
   sub2       f1    f2    f3
      sub21   f1    f2    f3
   sub3       f1    f2    f3

I want:

top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

Then I want to just load this up in Excel (delimited by whitespace) and still be able to look at the hierarchy-ness of the first column!

I tried many things, but end up losing the hierarchy information

5 Answers

With this as the input:

$ cat file
top           f1    f2    f3
   sub1       f1    f2    f3
   sub2       f1    f2    f3
      sub21   f1    f2    f3
   sub3       f1    f2    f3

Try:

$ sed -E ':a; s/^( *) ([^ ])/\1.\2/; ta' file
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

How it works:

  • :a

    This creates a label a.

  • s/^( *) ([^ ])/\1.\2/

    If the line begins with spaces, this replaces the last space in the leading spaces with a period.

    In more detail, ^( *) matches all leading blanks except the last and stores them in group 1. The regex ([^ ]) (which, despite what stackoverflow makes it look like, consists of a blank followed by ([^ ])) matches a blank followed by a nonblank and stores the nonblank in group 2.

    \1.\2 replaces the matched text with group 1, followed by a period, followed by group 2.

  • ta

    If the substituted command resulted in a substitution, then branch back to label a and try over again.

Compatibility:

  1. The above was tested on modern GNU sed. For BSD/OSX sed, one might or might not need to use:

    sed -E -e :a -e 's/^( *) ([^ ])/\1.\2/' -e ta file
    

    On ancient GNU sed, one needs to use -r in place of -E:

    sed -r ':a; s/^( *) ([^ ])/\1.\2/; ta' file
    
  2. The above assumed that the spaces were blanks. If they are tabs, then you will have to decide what your tabstop is and make substitutions accordingly.

There are two different ways to do this in vim.

  1. With a regex:

    :%s/^\s\+/\=repeat('.', len(submatch(0)))
    

    This is fairly straightforward, but a little verbose. It uses the eval register (\=) to generate a string of '.'s the same length as the number of spaces at the beginning of each line.

  2. With a norm command:

    :%norm ^hviwr.
    

    This is a much more conveniently short command, although it's a little harder to understand. It visually selects the spaces at the beginning of a line, and replaces the whole selection with dots. If there is no leading space, the command will fail on ^h because the cursor attempts to move out of bounds.

    To see how this works, try typing ^hviwr. on a line that has leading spaces to see it happen.

Since you said python:

#!/usr/bin/env python
import re, sys
for line in sys.stdin:
    sys.stdout.write(re.sub('^ +', lambda m: len(m.group(0)) * '.', line))

(for each line, we replace the longest run of prefix spaces '^ +' with an equally long string of dots, len(m.group(0)) * '.').

With the end result:

$ ./dottify.py <file
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

Since you said awk:

$ awk '{ match($0,/^ +/); p=substr($0,0,RLENGTH); gsub(" ",".",p); print p""substr($0,RLENGTH+1) }' file
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

(where for each line we match the longest prefix of spaces with match, extract it with substr, replace each space with dot via gsub, and print that modified prefix p, followed by the remainder of the input line (RSTART and RLENGTH variables are populated after match() and hold the starting position and length of the matched pattern).

In awk. It keeps replacing the first space with a period while the space is preceeded only by periods:

$ awk '{while(/^\.* / && sub(/ /,"."));}1' file
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

and here's one in perl:

$ perl -p -e 'while(s/(^\.*) /\1./){;}' file
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

A little lengthy, but a fun exercise nonetheless:

# Function to count the number of leading spaces in a string
# Basically, this counts the number of consecutive elements that satisfy being spaces
def count_leading_spaces(s):
    if not s:
        return 0
    else:
        curr_char = s[0]
        if curr_char != ' ':
            return 0
        else:
            idx = 1
            curr_char = s[idx]
            while curr_char == ' ':
                idx += 1
                try:
                    curr_char = s[idx]
                except IndexError:
                    return idx
        return idx

Finally, open up the file and do some work:

with open('file.txt', 'r') as f:
    data = []
    for i, line in enumerate(f):
        # Don't do anything to the field names
        if i == 0:
            new_line = line.rstrip()
        else:
            n_leading_spaces = count_leading_spaces(line)
            # Impute periods for spaces
            new_line = ('.'*n_leading_spaces + line.lstrip()).rstrip()
        data.append(new_line)

Results:

>>> print('\n'.join(data))
top           f1    f2    f3
...sub1       f1    f2    f3
...sub2       f1    f2    f3
......sub21   f1    f2    f3
...sub3       f1    f2    f3

You could also do it this way, which is much simpler:

with open('file.txt', 'r') as f:
    data = []
    for i, line in enumerate(f):
        # Don't do anything to the field names
        if i == 0:
            new_line = line.rstrip()
        else:
            n_leading_spaces = len(line) - len(line.lstrip())
            # Impute periods for spaces
            new_line = line.lstrip().rjust(len(line), '.').rstrip()
        data.append(new_line)
Related