itext7: deferred signing not producing a valid signed pdf

Question

I'm trying to use itext7 to sign a pdf by getting the signature from an external entity. I must be missing something because the deferred pdf signing is not valid. Lets start with the code for deferred signing:

public byte[] GetDocHashFromPreparedDocToSign(string pathToOriginalPdf, string pathToPreparedToBeSignedPdf, List certificates) {
    var pdfSigner = new PdfSigner(new PdfReader(pathToOriginalPdf),
                                  new FileStream(pathToPreparedToBeSignedPdf, FileMode.Create),
                                  new StampingProperties());
    pdfSigner.SetFieldName(_signatureFieldname);


    var appearance = pdfSigner.GetSignatureAppearance();
    appearance.SetPageRect(new Rectangle(144, 144, 200, 100))
              .SetPageNumber(1)
              .SetCertificate(certificates[0]);

    var container = new ExternalBlankSignatureContainer(PdfName.Adobe_PPKLite, PdfName.Adbe_pkcs7_detached);
    pdfSigner.SignExternalContainer(container, 8192);
    



   byte[] sha256SigPrefix = { 0x30, 0x31, 0x30, 0x0d, 0x06, 0x09,
                                 0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01,
                                 0x05, 0x00, 0x04, 0x20 };
    // service needs to receive sha256 prepended
    using var stream = File.OpenRead(pathToPreparedToBeSignedPdf);
    var data = DigestAlgorithms.Digest(stream, DigestAlgorithms.SHA256);        
    var totalHash = new byte[sha256SigPrefix.Length + data.Length];
    sha256SigPrefix.CopyTo(totalHash, 0);
    data.CopyTo(totalHash, sha256SigPrefix.Length);
    return totalHash;
}

The method received the path to the original pdf, the path to the temporary pdf that will contain the placeholder for the signature and a list of X509Certificate that is retrieved from the original service. After reserving the space for the signature, the method calculates the file's hash and prepends it with the sha256 prefix (required by the service that will sign the document).

This information is sent to the service which will return the signature. When the signature is retrieved, the following method is called for filling the signature placeholder with the real signature:

public void SignPreparedToBeSignedDoc(string pathToPreparedToBeSignedPdf, string pathToSignedFile, byte[] signature) {
    var document = new PdfDocument(new PdfReader(pathToPreparedToBeSignedPdf));
    using var writer = new FileStream(pathToSignedFile, FileMode.Create);

    var container = new ExternalInjectingSignatureContainer(signature);
    PdfSigner.SignDeferred(document, _signatureFieldname, writer, container);
}

EDIT: based on @mkl comment, I've fixed the signing part: And here's the ExternalInjectingSignatureContainer:

internal class ExternalInjectingSignatureContainer : IExternalSignatureContainer {
    private readonly byte[] _signature;

    public ExternalInjectingSignatureContainer(byte[] signature) {
        _signature = signature;
    }

    public byte[] Sign(Stream data){
        var sgn = new PdfPKCS7(null, _certificates.ToArray(), "SHA256", false);
        sgn.SetExternalDigest(_signature, null, "RSA");
        return sgn.GetEncodedPKCS7();

    }

    public void ModifySigningDictionary(PdfDictionary signDic) {
    }
}

Even though the code runs without errors, opening the pdf in adobe shows the following error:

EDIT: after fixing the signing code, now the error is different: it will show the signature info but it will say that the file has been changed or is corrupted.

At this point, it seems like the temporary pdf is being generated correctly, but I'm probably missing something...any clues on how I might debug this issue?

Thanks

EDIT: in response to the comments to the solution presented by @mkl, I've tried to update the code. I had a couple more minutes to play with this today and I've tried to follow the presented guidelines, but I'm clearly still missing something.

Before showing the new code, I'd just like to point out that the previous updated version (that whas using 2 IExternalSignatureContainer instances) seemed to be working correctly. ie, opening the signed pdf on adobe would only show me the yellow warning saying that there was something wrong with the signature:

Since the doc is being used with a test chain, it seems like the signing worked ok (though I might be completely wrong).

So, in order to fix the incorrect usage of the container, I've rewritten the code for the IExternalSignatureContainer's Sign methods. Here's the code I've got for the one that prepares the document hash that is going to be sent to the server:

public override byte[] Sign(Stream data) {
// create PCKS7 for getting attributes
var sgn = new PdfPKCS7(null, _certificates.ToArray(), DigestAlgorithms.SHA256, false);
// get document hash
DocumentDigest = DigestAlgorithms.Digest(data, DigestAlgorithms.SHA256);
// get attributes
var docBytesHash = sgn.GetAuthenticatedAttributeBytes(DocumentDigest, 
                                PdfSigner.CryptoStandard.CMS, 
                                null, 
                                null);
 //prepend sha256 prefix
 var totalHash = new byte[_sha256SigPrefix.Length + docBytesHash.Length];
 _sha256SigPrefix.CopyTo(totalHash, 0);
 docBytesHash.CopyTo(totalHash, _sha256SigPrefix.Length);
 DataToSend = totalHash;
 return new byte[0];
}

Since I must call the GetEncodedPKCS7 method with the same parameters that were passed to GetAuthenticatedAttributes, I'm also saving the document hash obtained through the Digest method. DataToSend will be sent to the server so that it can return the signature for that hash.

And here's the code for the other IExternalSignatureContainer that will be called for the deferred signing (PdfSigner.SignDeferred):

public byte[] Sign(Stream data) {
    // create CMS      
    var sgn = new PdfPKCS7(null, _certificates.ToArray(), DigestAlgorithms.SHA256, false);
    // set the signature bytes
    sgn.SetExternalDigest(_signature, null, "RSA");
    // call GetEncoded with the same parameters as the original GetAuthenticatedAtt...
    //_documentHash == DocumentDigest previous sample
    var encodedSig = sgn.GetEncodedPKCS7(_documentHash,
                                         PdfSigner.CryptoStandard.CMS,
                                         null,
                                         null,
                                         null);

    return encodedSig;
}

Unfortunately, I must be missing something (or lots of things):

Did I completely missed your point?

EDIT: Once again, following @mkl's lead, I was able to make it work. Like he said, you need to hash the GetAuthenticatedAttributeBytes value:

public override byte[] Sign(Stream data) {
// create PCKS7 for getting attributes
var sgn = new PdfPKCS7(null, _certificates.ToArray(), DigestAlgorithms.SHA256, false);
// get document hash
DocumentDigest = DigestAlgorithms.Digest(data, DigestAlgorithms.SHA256);
// get attributes
var docBytes = sgn.GetAuthenticatedAttributeBytes(DocumentDigest, 
                                PdfSigner.CryptoStandard.CMS, 
                                null, 
                                null);
// hash dochBytes 
using var hashMemoryStream = new MemoryStream(docBytes, false);
var docBytesHash = DigestAlgorithms.Digest(hashMemoryStream, 
                                            DigestAlgorithms.SHA256);
 //prepend sha256 prefix
 var totalHash = new byte[_sha256SigPrefix.Length + docBytesHash.Length];
 _sha256SigPrefix.CopyTo(totalHash, 0);
 docBytesHash.CopyTo(totalHash, _sha256SigPrefix.Length);
 DataToSend = totalHash;
 return new byte[0];
}

Thanks again.

user_1729265 · Answer

There are two apparent issues, you hash the wrong data and you inject a wrong type of signature:

Hashing the wrong data

You calculate the hash to be signed like this:

using var stream = File.OpenRead(pathToPreparedToBeSignedPdf);
var data = DigestAlgorithms.Digest(stream, DigestAlgorithms.SHA256);

This is not correct.

A signed PDF essentially has this structure (read here for more details):

(By the way, the sketch is not 100% correct as the angled bracket delimiters '<' and '>' around the signature value must also not be hashed.)

Your prepared PDF at pathToPreparedToBeSignedPdf has the same structure, merely the "signature value" is not yet an actual signature value but instead a placeholder of 8192 hex-encoded zero bytes (8192 because that's the number you gave in pdfSigner.SignExternalContainer).

As you can see in the sketch, though, the signature value (or in your case, the placeholder) must not be hashed for signing.

The easiest way to retrieve the prepared PDF except the placeholder is inside the IExternalSignatureContainer implementation you use for preparing the PDF as its Sign method as parameter gets a stream containing exactly that. So instead of the ExternalBlankSignatureContainer use something like the ExternalEmptySignatureContainer from this answer:

public class ExternalEmptySignatureContainer : IExternalSignatureContainer
{
    public void ModifySigningDictionary(PdfDictionary signDic)
    {
        signDic.Put(PdfName.Filter, PdfName.Adobe_PPKLite);
        signDic.Put(PdfName.SubFilter, PdfName.Adbe_pkcs7_detached);
    }

    public byte[] Sign(Stream data)
    {
        // Store the data to sign and return an empty array
        Data = DigestAlgorithms.Digest(data, DigestAlgorithms.SHA256);
        return new byte[0];
    }

    public byte[] Data;
}

and after preparing retrieve the byte[] from its Data member:

var container = new ExternalEmptySignatureContainer();
pdfSigner.SignExternalContainer(container, 8192);
byte[] hash = container.Data;

In your case you may have to prepend the sha256SigPrefix to get a complete encoded DigestInfo object.

Injecting a wrong type of signature

Furthermore, considering your screen shot

you apparently inject a signature of a wrong type. You set a subfilter PdfName.Adbe_pkcs7_detached which implies that the signature to embed is a CMS signature container with a single SignerInfo signing the signed bytes of the PDF. The error message indicates, though, that the signature container you embed either is broken or is not a CMS signature container to start with.

After your edit: Injecting a signature container with incorrect contents

To fix the previous issue, "Injecting a wrong type of signature", you changed your ExternalInjectingSignatureContainer to build a CMS signature container for your signature bytes like this:

var sgn = new PdfPKCS7(null, _certificates.ToArray(), "SHA256", false);
sgn.SetExternalDigest(_signature, null, "RSA");
return sgn.GetEncodedPKCS7();

This, unfortunately, uses the PdfPKCS7 class incorrectly resulting in a CMS signature container with incorrect data.

(Correction: The resulting CMS container is not incorrect per se, but it is structurally extremely simply. Nowadays many profiles - e.g. European PAdES - require additional, signed attributes, and to satisfy such profiles you have to fix as described in the following text. If you only need Adobe Reader to show valid, though, the code above suffices for you.)

To fix you can either indeed build a CMS signature container yourself, using the iText PdfPKCS7 class or other means (like BouncyCastle or built-in .Net classes), or you can let iText do that for you.

The easiest approach is the latter one. You actually don't need deferred signing here at all, you simply implement IExternalSignature so that it calls your remote service:

public class RemoteSignature : IExternalSignature
{
    public virtual byte[] Sign(byte[] message) {
        IDigest messageDigest = DigestUtilities.GetDigest(GetHashAlgorithm());
        byte[] messageHash = DigestAlgorithms.Digest(messageDigest, message);
        byte[] digestInfo = [... prefix messageHash with sha256SigPrefix ...];
        //
        // Request signature for DigestInfo digestInfo 
        // and return signature bytes
        //
        return CALL_YOUR_SERVICE_FOR_SIGNATURE_OF_HASH(digestInfo);
    } 

    public virtual String GetHashAlgorithm() {
        return "SHA-256";
    } 

    public virtual String GetEncryptionAlgorithm() {
        return "RSA";
    } 
}

Now you can simply sign doing

var pdfSigner = new PdfSigner(new PdfReader(pathToOriginalPdf),
                              new FileStream(pathToSignedFile, FileMode.Create),
                              new StampingProperties());
pdfSigner.SetFieldName(_signatureFieldname);

var appearance = pdfSigner.GetSignatureAppearance();
appearance.SetPageRect(new Rectangle(144, 144, 200, 100))
          .SetPageNumber(1)
          .SetCertificate(certificates[0]);

var signature = new RemoteSignature();
pdfSigner.SignDetached(signature, certificates, null, null, null, 0, PdfSigner.CryptoStandard.CMS);

itext7: deferred signing not producing a valid signed pdf

1 Answers

Hashing the wrong data

Injecting a wrong type of signature

After your edit: Injecting a signature container with incorrect contents