Extracting image bounding box and arrows from document page using OpenCV

Hi i'm struggling with an algorithm that can effectively extract image bounding box and arrows from a rasterized document. The arrows and images can change is size shape and color. The arrows may not always be arrows but rather lines. The images may be just an outline or a full color picture. This is the code i wrote so far and it kinda works but not always. I wrote this algorithm based on the excellent paper Image to CAD: Feature Extraction and Translation of Raster Image of CAD Drawing to DXF CAD Format by Aditya Intwala.

This is the original image:

For detecting the arrowheads

 using var kernel1 = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(2, 2));
 using var binary = grayImage.Threshold(0, 255, ThresholdTypes.Binary | ThresholdTypes.Otsu);
 using var invertedBinary = grayImage.Threshold(0, 255, ThresholdTypes.BinaryInv | ThresholdTypes.Otsu);
 using var blackHatResult = binary.MorphologyEx(MorphTypes.BlackHat, kernel1);
 using var solidArrowHeads = invertedBinary - blackHatResult;
 using var foundArrowHeads = solidArrowHeads.ToMat();
 using var steKernel = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));
 using var eroded = foundArrowHeads.Erode(steKernel);
 using var dialated = eroded.Dilate(steKernel); 
 dialated.FindContours(out var arrowHeadContours, out var hierarchy, RetrievalModes.External,
     ContourApproximationModes.ApproxSimple);

After finding arrow heads I;m doing the following for finding the lines that intersect the bounding box of arroheads. These intersecting lines is what im classifying as an arrow. It does work but i get a lot more false positives and would like to know how to improve my algorithm or if theres a better way. Next i'm erasing the arrows by masking it and then using this image for the next step.

To find the boundaries of the images I've written the following code

using var se1 = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(70, 1));
using var closedImg = grayImage.MorphologyEx(MorphTypes.Close, se1);
Cv2.BitwiseAnd(grayImage, closedImg, grayImage);
using var structuringElement = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(15, 15));
using var blurred = grayImage.MorphologyEx(MorphTypes.Gradient, structuringElement);
using var inverted = blurred.Threshold(197, 255, ThresholdTypes.Binary);

//Lets get bounding boxes for all large contours
Cv2.FindContours(inverted, out var contours, out var hierarchyIndices, RetrievalModes.External,
    ContourApproximationModes.ApproxSimple);

for (var i = 0; (i >= 0) && (i < hierarchyIndices.Length); i = hierarchyIndices[i].Next)
{
    CT_Assert.True(hierarchyIndices[i].Parent == -1, "Must be a top level contour ?");

    var rect = Cv2.BoundingRect(contours[i]);
    // save this rect as possible bounding box
}

This again kinda works but not always.

Final output image, black bounding box is detected arrowhead, red lines are arrows and blue boxes are detected image bounding boxes.

Final Output image: