Converting a the outlines of a free drawn path to a polygon

I want to segment multiple objects within an image using the fabric.js javascript library.

For this example, I want to segment the surfboard. I then want want to convert the drawn shape to a polygon with the coordinates of the outer regions corresponding to the pixel coordinates of the images.

Drawing an segmentation itself it quite straightforward using fabric.js's build in free drawing brush.

I can then convert the coordinates of the drawn path to a polygon using this piece of code:

canvas.on('path:created', function(el){
    var path = el.path.path
  var points = []
  for (var i = 0; i < path.length; i++){
    point = {
      x: Math.round(path[i][1]),
      y: Math.round(path[i][2])
    }
    points.push(point)
  }

  shape = new fabric.Polygon(points, {
      stroke: 'red',
      opacity: 0.5,
      strokeWidth: 1,
      description: 'aaa',
      fill: 'transparent',
  });
  canvas.add(shape)
})

The red/black link is how I was drawing using the mouse.

After turning drawing mode off and dragging them away, the 2 created objects (the Path from drawing and the polygon) look as follows.

However, because the brush has a certain width, and because of overlapping Path regions, my method of converting it to a poly is not working well.

So I don't want the current polygon I'm outputting (the red one), but instead the outline of the green section. How can I achieve this?

Working fiddle: https://jsfiddle.net/haw54v9L/3/