How to create an edge preserving blur (similar to a bilateral filter) using a limited set of primitive operations

I have been attempting to duplicate a bilateral filter effect (edge preserving, color-range aware) using the limited set of primitives in the existing toolbox of SVG filters. I've tried a number of approaches. My most successful to date is a three part operation that does Sobel edge detection, dilates the Sobel edges, extracts the pixels corresponding to those edges with a compositing operation, gaussian blurs the source image and then composites back the original edge pixels on top of the blurred image. The result preserves edges, but is not color-range aware.

<filter id="surfaceBlur" color-interpolation-filters="sRGB">
        <!-- convert source image to luminance map-->
        <feColorMatrix type="luminanceToAlpha" />
        <!-- sober edge detection-->
        <feConvolveMatrix order="3" kernelMatrix="-1 -2 -1  
                                                    0 0 0  
                                                   1 2 1 "
                          preserveAlpha="true"
                         />
        <feConvolveMatrix order="3" kernelMatrix="-1 0 1  
                                                  -2 0 2 
                                                  -1 0 1 "
                          preserveAlpha="true"
                          />
      <!-- dilate the edges to produce a wider mask-->
      <feMorphology operator="dilate" radius="1"
                     result="mask"/>
      <!-- extract just the detail from the source graphic using the dilated edges -->
      <feComposite operator="in" in="SourceGraphic" in2="mask" result="detail" />
      <!-- blur the source image -->
      <feGaussianBlur stdDeviation="3" in="SourceGraphic" result="backblur"/>
       <!-- slap the detail back on top of the blur! -->
      <feComposite operator="over" in="detail" in2="backblur"/>

You can see the original, a gaussianBlur, this filter and in the bottom right, a true bilateral filter:

http://codepen.io/mullany/details/Dbyxt

As you can see, it's not an awful result, but it's not very close to a bilateral filter. This method also only works with greyscale images because it uses luminance differences to find edges - so edges between colors of similar luminance are not detected.

So the question is whether there is a algorithm variant of an edge preserving color range aware filter (guided edge view, bilateral etc.) - that can be constructed using the limited primitives available in SVG - which for those not familiar with SVG are:

• gaussian blur
• convolution (any kernel size)
• erode/dilate
• color matrix
• all porter duff compositing operations
• basic blending ops (multiply, screen, lighten, darken)
• a component transfer primitive that allows color channels to be transformed with a table lookup (as well as floored/ceilinged to specific values)

Only the RGB color space is available. Multiple iterations are fine, and any directed graph of these operations can be constructed.

Update:

I successfully created a median filter using feBlend lighten and darken as Max and Min operators in a bubble sort (thanks to help from cs.stackexchange.com). However this is inefficient: http://codepen.io/mullany/pen/dmbvz, and lacks the color range awareness of a bilateral filter.