D3: use nest function to turn flat data with parent key into a hierarchy

I'm sure there's a really simple elegant way to do this but I can't quite figure it out. I have some input data that looks like this:

[
{id: 1, name: "Peter"},
{id: 2, name: "Paul", manager: 1},
{id: 3, name: "Mary", manager: 1},
{id: 4, name: "John", manager: 2},
{id: 5, name: "Jane", manager: 2}
]

If possible, I would like to use the d3.js nest operator to get a structure to use in the hierarchy layout. Like this:

[ 
   {name: "Peter", children: [
          {name:"Paul", children: [
              {name:"John"},
              {name:"Jane"}
          ]},
          {name:"Mary"}
      ]
   }
]