Build multidimensional array using nested foreach loops

I am attempting to build an multidimensional array using nested foreach loops.

The data I am working with looks like this. category_ID, category_parent_ID, category_type

First I want to build seperate arrays based on the value contained in category_type.

foreach( $car_taxonomies as $car_taxonomy ){
  if( $car_taxonomy['car_taxonomy_type'] == 'Make' ){
    $makes[] = $car_taxonomy;
  }
  if( $car_taxonomy['car_taxonomy_type'] == 'Model' ){
    $models[] = $car_taxonomy;
  }
  if( $car_taxonomy['car_taxonomy_type'] == 'Badge' ){
    $badges[] = $car_taxonomy;
  }
}

Now I have $makes, $models and $badges.

What I want my desired array to look like is...

array(
  array(
    'category_ID' => 1,
    'category_parent' => 0,
    'category_type' => 'Make',
    'children' => array(
      array(
        'category_ID' => 20,
        'category_parent' => 1,
        'category_type' => 'Model',
        'children' => array(
          array(
            'category_ID' => 1,
            'category_parent' => 20,
            'category_type' => 'Badge',
          )
        )
      )
    )
  )
  array(
    'category_ID' => 4,
    'category_parent' => 0,
    'category_type' => 'Make',
    'children' => array(
      array(
        'category_ID' => 17,
        'category_parent' => 4,
        'category_type' => 'Model',
        'children' => array(
          array(
            'category_ID' => 34,
            'category_parent' => 17,
            'category_type' => 'Badge',
          )
        )
      )
    )
  )
)

My next step doesnt seem to work completely especially on the most inner foreach loop. But what I have is the following. What ever it is something doesnt seem to work completely to return my desired array.

foreach( $makes as $makekey => $make ){
  $tree[$makekey] = $make;
  foreach( $models as $modelkey => $model ){
    if( $model['car_taxonomy_parent'] == $make['car_taxonomy_ID'] ){
      $tree[$makekey]['children'][] = $model;
      foreach( $badges as $badgekey => $badge ){
        if( $badge['car_taxonomy_parent'] == $model['car_taxonomy_ID'] ){
          $tree[$makekey][$modelkey]['children'][] = $badge;
        }
      }
    }
  }
}

return $tree;

Any help on this would be greatly appreciated.