PDF Signing, generated PDF Document certification is invalid? (using external signing, web-eid, HSM)

I have a service which signs the data and provides me with the signed hash, it correctly generates PKCS#7 DigestInfo as stated in rfc2315#section-9.4

Something like this

The code for the above system is : https://pastebin.com/b3qZH6xW

            //prepare signature
        PDSignature signature = new PDSignature();
        signature.setFilter(PDSignature.FILTER_ADOBE_PPKLITE);
        signature.setSubFilter(PDSignature.SUBFILTER_ADBE_PKCS7_DETACHED);
        signature.setName("Ankit");
        signature.setLocation("Bhopal, IN");
        signature.setReason("Testing");
        // TODO extract the above details from the signing certificate? Reason as a parameter?

        // the signing date, needed for valid signature
        signature.setSignDate(Calendar.getInstance());

        if (accessPermissions == 0)
        {
            setMDPPermission(document, signature, 3);
        }

        FileOutputStream fos = new FileOutputStream(new File("signed_file.pdf"));

        DetachedPkcs7 detachedPkcs7 = new DetachedPkcs7();
        //populate signature options for visible signature. if any.
        SignatureOptions signatureOptions = null;
        document.addSignature(signature);
        ExternalSigningSupport externalSigning = document.saveIncrementalForExternalSigning(fos);
        InputStream dataToSign = externalSigning.getContent();
        byte[] cmsSignature = detachedPkcs7.sign(dataToSign);
        externalSigning.setSignature(cmsSignature);  

Workflow is something like this
- Grab original PDF
- add signature dictionary and get the hash
- send the hash to client
- Wait for data on Standard Input.
- Wait for Client to send the signed hash back, This data is then feeded to the paused program, that is, the data is sent to standard input of the program
- add the CMS. :)

I have no clue why the PDF generated using this process has the signature shown as invalid.