How can I do join with whereDoesntHave in Laravel

I'm trying to run a query in my Laravel 8.0 application. I've a problem doing join (circular) using the concept of whereDoesntHave() in Laravel. Since I'm using DB::table('...') and it has other conditions implemented, I'm having a brands table, projects table and project_associate_brand table, brand is in many-to-many relation with project, I want to find out the project counts in which brand and projects are not associated with each other, I'm trying to join the table twice one for getting projects associated with and other one to remove where it is not in relationship, so my query looks like:

$innerQuery = DB::table('brands as b')
    ->join('project_associate_brand as  pabdoesnthave', 'b.id', '=', 'pabdoesnthave.brand_id')
    ->join('project_associate_brand as pab', function ($join) {
        $join->on('b.id', '!=', 'pab.brand_id')
            ->where('pabdoesnthave.project_id', '!=', 'pab.project_id');
    })
    ->join('projects as p', function ($join) {
        $join->on('pab.project_id', '=', 'p.id')
           ->where('pabdoesnthave.project_id', '!=', 'p.id');
    })
     ->select(DB::raw('b.id, b.title, COUNT(DISTINCT p.id) as projects_count'))
    ->whereNull('b.deleted_at')
    ->whereNull('p.deleted_at')
    ->groupByRaw('b.id');
    
    
return ProductBrandFilterResource::collection(DB::query()->fromSub($innerQuery, 't')
    ->select(DB::raw("
                id,
                `title`,
                projects_count
         "))
    ->orderBy('title', 'asc')
    ->groupBy('t.id')->get());

I'm unable get the desired results. To explain better I've table porject_associate_brand as:

brand_id    project_id
    1           1
    1           2
    1           3
    2           1
    2           3

If I want to get the list of brand where projects are not associated I get

id  title   project_count
1   ABC         3
2   PQR         3

For id 2 it is giving me 3 but actually it should be 1 as result.

Any better approach is appreciated. Thanks