cdkDrag sometimes grabs uderlying link element

I have the cdkList as a parent component and it has a child where the elements are wrapped in a tag. My issue is that sometimes when I drag the element the underlying link will be grabbed instead of dragging/grabbing the child element/component (pic below). When I change the tag to div or the issues goes away. As these should act as links I can't do that. I also tried with cdkDragHandle and with that the issue was even more frequent.

Here is the parent:

<div (cdkDropListDropped)="favouritesService.drop($event, favourites)" cdkDropList cdkDropListOrientation="vertical">
    <app-favourites
        *ngFor="let favourite of favourites"
        [favouriteName]="favourite.mainFolderName || setFavouriteName(favourite)"
        [favourite]="favourite"
        [stackB64Image]="dataService.getB64Image(stacksService.findStack(favourite.stackId || '')?.thumb_b64)"
        [stackThumbImage]="
            favourite.stackId
                ? stacksService.findStack(favourite.stackId).processedThumbPathGcs ||
                  stacksService.findStack(favourite.stackId).thumbImagePathGcs
                : undefined
        "
        [thumbnailImage]="dataService.setFavouriteThumbnail(favourite)"
    ></app-favourites>
</div>

Here is the child:

<a
    (click)="handleClick()"
    [queryParams]="{
        integrationId: favourite['integrationId'],
        integrationName: favourite['integrationName'],
        conversationId: favourite['conversationId'],
        conversationName: favourite['conversationName'],
        folderId: favourite['folderId'],
        folderName: favourite['folderName'],
        stackId: favourite['stackId'],
        stackName: favourite['stackName'],
        favCatFilter: favourite['favoriteCategoryFilter']
    }"
    [routerLinkActiveOptions]="{ exact: true }"
    [routerLink]="dataService.getRouterLink(favourite)"
    cdkDrag
    class="flex items-center outline-none text-text font-Montserrat-Medium text-sm w-66 h-10 cursor-pointer rounded hover:bg-white-10 focus:outline-none"
    routerLinkActive="rounded bg-black-30"
>
    <img
        [src]="stackB64Image || (stackThumbImage ? (stackThumbImage | getDownloadURL) : thumbnailImage)"
        [style.border-radius]="favourite['stackId'] ? '3px' : ''"
        alt="{{
            favourite['integrationSource'] ||
            favourite['itemType'] ||
            favouriteName ||
            favourite['favoriteCategoryFilter']
                ? favourite['favoriteCategoryFilter']
                : '' + ' logo'
        }}"
        style="margin-left: 15px; margin-right: 9px; width: 26px; height: auto; max-height: 26px; object-fit: cover"
    />
    <p class="cursor-pointer font-Montserrat-Medium text-left text-sm truncate" style="width: 206px">
        {{ favouriteName }}
    </p>
</a>

Picture about the issue: