How use openssl to verify a certificate in PEM with a local trust store?

The overall OpenSSL documentation around this topic is rather limited and has broken links all over the place, so my approach might not the only one or best one. As far as I can see, verifying a certificate (chain) is done with the following steps, unrolled in reverse order because I think that gives a better understanding. For the resulting code, see the end of this answer. All code has error checking omitted for the sake of brevity. Also, the loading of Certificate Revocation Lists (CRLs) is not explained, I think that is beyond the scope of your question.

The actual verification function

The functionality to verify a certificate (chain) is provided by the OpenSSL function X509_verify_cert(). A return value of 1 indicates successful verification, 0 indicates no success. As you can see in the documentation, the function only requires one parameter of the type X509_STORE_CTX, which is a structure holding the "context" (a rather vague and overused term in OpenSSL, IMO) of the collection of X509 certificates involved.

Setting up the certificate store context

The certificate store context contains information about trusted certificates, untrusted intermediate certificates and the certificate to be verified. It is constructed and initialized as follows:

store_ctx = X509_STORE_CTX_new();
X509_STORE_CTX_init(store_ctx, store, cert, intermediates)

The store parameter will be used to contain information about the trusted certificates, the cert parameter contains the certificate to be verified and the and intermediates parameter is a stack of untrusted intermediate certificates.

The store parameter

The X509_STORE type is able a contain a set of X509 certificates and for the purpose of verifying a certificate needs to be provided with information about trusted certificates. Since you indicated that you have trusted certificates in /etc/ssl/certs, this can be done as follows:

store = X509_STORE_new();
lookup = X509_STORE_add_lookup(store, X509_LOOKUP_hash_dir());
X509_LOOKUP_add_dir(lookup, "/etc/ssl/certs", X509_FILETYPE_PEM);

This assumes that your local trust store is set up properly

The cert parameter`

This parameter contains the actual certificate to be verified. It can be loaded from a file in several ways, one approach is as follows:

bio_in = BIO_new_file(certFileName, "r");
result = PEM_read_bio_X509(bio_in, NULL, NULL, NULL);
BIO_free(bio_in);

The intermediates parameter

OpenSSL provides a stack API to handle collections of objects. The intermediates parameter is a stack of X509 objects that contains the intermediate certificates between your certificate to be tested and your trusted certificate. In pseudo code, it can be filled as follows:

intermediates = sk_X509_new_null();
for (filename in certFilenames) do {
    icert = readCert(filename);
    sk_X509_push(intermediates, icert);
}

This concludes the explanation, this should give you all you need to verify the chain.

**About the certificate at the end of the downloaded chain

The certificate at the end of the downloaded chain is typically contained in your local trust store. Some experiments show that you can actually feed it into the verify function as if it is an untrusted intermediate or you can omit it. Both seemed to end in a properly verified chain.

A code example

Finally :-)

#include <openssl/x509.h>
#include <openssl/x509_vfy.h>
#include <openssl/pem.h>

const char *trustedCertsPath = "/etc/ssl/certs";

int main(
    int argc,
    char **argv)
{
    X509 *cert = NULL;
    X509 *icert = NULL;
    STACK_OF(X509) *intermediates = NULL;
    X509_STORE *store = NULL;
    X509_LOOKUP *lookup = NULL;
    X509_STORE_CTX *store_ctx = NULL;
    BIO *bio_in = NULL;
    int currentArg = 1;
    int result = 0;

    store = X509_STORE_new();
    lookup = X509_STORE_add_lookup(store, X509_LOOKUP_hash_dir());
    X509_LOOKUP_add_dir(lookup, trustedCertsPath, X509_FILETYPE_PEM);

    /* Certificate to be checked */
    bio_in = BIO_new_file(argv[currentArg++], "r");
    cert = PEM_read_bio_X509(bio_in, NULL, NULL, NULL);
    BIO_free(bio_in);

    /* Stack of untrusted intermediate certificates */
    intermediates = sk_X509_new_null();
    while (currentArg < argc) {
        bio_in = BIO_new_file(argv[currentArg++], "r");
        icert = PEM_read_bio_X509(bio_in, NULL, NULL, NULL);
        BIO_free(bio_in);
        sk_X509_push(intermediates, icert);
    }

    store_ctx = X509_STORE_CTX_new();
    X509_STORE_CTX_init(store_ctx, store, cert, intermediates);

    result = X509_verify_cert(store_ctx);
    printf("Result from X509_verify_cert is %d\n", result);

    sk_X509_pop_free(intermediates, X509_free);
    X509_STORE_CTX_cleanup(store_ctx);
    X509_STORE_CTX_free(store_ctx);
    X509_STORE_free(store);
}

You can build and run it as follows (where the .pem arguments are the names of the files containing your certificate and intermediates in PEM format:

$ gcc  main.c $(pkg-config openssl --libs) -o verify -Wall
$ ./verify \*.google.com.pem Google\ Internet\ Authority\ G3.pem
Result from X509_verify_cert is 1