How to create sub array and group the same data?

I use PHP Laravel.

I'm having trouble displaying JSON from sub array data with groupBy grade or point. This is a refinement of the my previous question.

My Table

+----+---------+---------+------------+------------+-------------+-------+-------+
| id | club_id | user_id | contact_id | product_id | category_id | grade | point |
+----+---------+---------+------------+------------+-------------+-------+-------+
| 1  | 1       | 1001    | 10083      | 1          | 21          | A     | 100   |
+----+---------+---------+------------+------------+-------------+-------+-------+
| 2  | 1       | 1001    | 10083      | 2          | 22          | A     | 100   |
+----+---------+---------+------------+------------+-------------+-------+-------+
| 3  | 1       | 1001    | 10083      | 1          | 23          | B     | 80    |
+----+---------+---------+------------+------------+-------------+-------+-------+
| 4  | 1       | 1001    | 10083      | 2          | 24          | B     | 80    |
+----+---------+---------+------------+------------+-------------+-------+-------+
| 5  | 2       | 2299    | 87655      | 1          | 27          | A     | 100   |
+----+---------+---------+------------+------------+-------------+-------+-------+

My Code:

$data = Supplier::where('club_id', 1)->get();

if ($data->isEmpty()) {
    return [];
}

$data = $data->map(function($item) use($data) {
    $detail = $data->where('user_id',$item['user_id']);
    $detail = collect($detail)->map(function($item) {
        return collect($item)->only(['id','product_id','category_id']);
    });
    $item['detail'] = $detail;
    return $item;
});

return [
    'supplier' => $data
];

My Json what I want: URL: http://json-parser.com/bfbd6302/3

{
    "supplier": [{
            "user_id": 1001,
            "contact_id": 10083,
            "grade": "A",
            "point": 100,
            "detail": [{
                    "id": 1,
                    "product_id": 1,
                    "category_id": 21
                },
                {
                    "id": 2,
                    "product_id": 2,
                    "category_id": 22
                }
            ]
        },
        {
            "user_id": 1001,
            "contact_id": 10083,
            "grade": "B",
            "point": 80,
            "detail": [{
                    "id": 3,
                    "product_id": 1,
                    "category_id": 23
                },
                {
                    "id": 4,
                    "product_id": 2,
                    "category_id": 24
                }
            ]
        }
    ]
}

My current result JSON: URL: http://json-parser.com/bfbd6302/4