Balanced braces/brackets/parens regex in Ruby

I came across a regex in the pickaxe Ruby book for finding balanced brace-expressions, and I'm trying to build upon it to get a regex that matches balanced braces/brackets/parens.

The original:

re = /
  \A
    (?<brace_expression>
      {
        (
          [^{}]                 # anything other than braces
        |                       # ...or...
          \g<brace_expression>  # a nested brace expression
        )*
      }
    )
  \Z
/x

My version so far:

re = /
  \A
    (?<brace_expression>
      (?:
        (?<brace> { ) | (?<bracket> \[ ) | ( \( )
      )
        (
          [^{}\[\]()]           # anything other than braces
        |                       # ...or...
          \g<brace_expression>  # a nested brace expression
        )*
      (?(<brace>) } | (?(<bracket>) \] | \) ) )
    )
  \Z
/x

It correctly matches "{xyz}", "[xyz]", "(xyz)", and correctly fails to match something like "{xyz]", but the recursion isn't behaving as I'd expect. It fails to match nested brace expressions like "{[]}". What am I missing?