How to convert SVG polylines to quickdraw stroke-3 numpy format?

Question

I would like to convert a basic SVG file containing polylines into the stroke-3 format used by sketch-rnn (and the quickdraw dataset).

To my understanding, each polyline point in stroke-3 format would be:

stored as [delta_x, delta_y, pen_up], where
delta_x, delta_y represent the coordinates relative to the previous point and
pen_up is a bit that is 1 when the pen is up (e.g. move_to operation a-la turtle graphics) or 0 when the pen is down (e.g. line_to operation a-la turtle graphics).

I've attempted to write the function and convert an SVG, but I when I render a test of the stroke-3 format I get an extra line.

My input SVG looks like this:

Here is a visualisation where the lines parsed from the SVG file are rendered in thick green and the lines drawn from the converted stroke-3 format are rendered in thinner red:

Notice the diagonal line on the right hand side face which isn't present in the original SVG.

I must be doing something wrong marking a line operation instead of a move operation somewhere, but I've been staring at the code for so long I can't spot the error.

This is a minimal example showing my attempt using svg.path:

import xml.etree.ElementTree as ET
import numpy as np
from svg.path import parse_path
from svg.path.path import Line, Move

cubeSVG = ''

def svg_to_stroke3(svg_string):
    # parse the doc
    doc = ET.fromstring(svg_string)
    # get paths
    paths = doc.findall('.//{http://www.w3.org/2000/svg}path')

    strokes = []
    # previous x, y
    px, py = 0, 0

    for path_index, path in enumerate(paths):
        stroke = parse_path(path.attrib['d'])
        was_moving = False
        for operation_index, operation in enumerate(stroke):
            if isinstance(operation, Move):
                mx  = int(operation.start.real)
                my  = int(operation.start.imag)
                # prep this end point for check as next line first point
                was_moving = True

                strokes.append([mx-px, my-py, 1])
                # update previous (absolute) coordinates
                px = mx
                py = my

            if isinstance(operation, Line):
                sx  = int(operation.start.real)
                sy  = int(operation.start.imag)
                ex  = int(operation.end.real)
                ey  = int(operation.end.imag)
                if was_moving:
                    # append delta x, y relative to previous move operation
                    strokes.append([sx-px, sy-py, 0])
                    was_moving = False
                # append delta x,y (line end relative to line start)
                strokes.append([ex-sx, ey-sy, 0])
                # update previous (absolute) coordinates
                px = ex
                py = ey

                # update previous end point
    
    strokes_np = np.array(strokes, dtype=np.int16)
    return strokes_np

print(svg_to_stroke3(cubeSVG))

Additionally I've made the above available as an easy to run Google Colab Notebook.

user_3944322 · Answer

Your conversion is correct, the bug is in the rendering code. It must be is_down = data[i][2] == 0 instead of is_down = data[i-1][2] == 0 in draw_stroke3.

This error didn't show up with the other paths as in all but two cases the new path starts at the end of the previous path. In the other case where you really move to a new start point the additional line coincided with a line already drawn.

UPDATE AND CORRECTION:
I noticed that I mis-interpreted the meaning of the pen-up bit: in fact it shows that the pen is to be lifted after drawing the current stroke, not for the current stroke as I though at first. Therefore your rendering code appears to be OK and the bug is in the stroke3 file generation.
I guess you can do it much simpler by recording the end points for each operation along with the op code (1 = move, 0 = draw) for the current operation. After conversion to a numpy array we can easily convert these absolute positions the relative displacements by do the difference of the first two columns and then shift the third column with the op codes backwards by one position:

def svg_to_stroke3(svg_string):
    doc = ET.fromstring(svg_string)
    paths = doc.findall('.//{http://www.w3.org/2000/svg}path')
    strokes = [[0,0,0]]
    for path in paths:
        stroke = parse_path(path.attrib['d'])
        for op in stroke:
            if isinstance(op, Move):
                strokes.append([op.start.real, op.start.imag, 1])
            if isinstance(op, Line):
                strokes.append([op.end.real, op.end.imag, 0])
    strokes_np = np.array(strokes, dtype=np.int16)
    # convert x,y columns to relative displacements
    strokes_np[1:,:2] -= strokes_np[:-1,:2]
    # shift back op codes
    strokes_np[:-1,2] = strokes_np[1:,2]
    # remove [0,0,0]s and set previous op to 0
    m = (strokes_np == 0).all(1)
    strokes_np[np.argwhere(m)-1,2] = 0
    return strokes_np[~m]

This gives a rather compact representation (49 rows for the example cube) and renders correctly with your original code and David Ha's draw_strokes() (you can do without the initial row I added if you by convention start with the pen up as draw_strokes() does).

How to convert SVG polylines to quickdraw stroke-3 numpy format?

1 Answers