Laravel 5.4: How to order by count(column) in eloquent?

I am trying to display the names of assignee arranged by the most tickets assigned_to.

$accesses = Access::where('state','=','Assigned');

$all = Report::where('state', '=', 'Assigned')
               ->union($accesses)
               ->orderBy('count(assigned_to)') //THIS IS WRONG
               ->get();