RSA_sign vs openssl command

Question

What's difference between calling RSA_sign from C++ code and calling

openssl dgst -sha256 -sign privkey.key -out hash.sig hash

from linux terminal? Why do I get different signature? My code is below.

static RSA* load_rsa_private_key(const char *rsa_key_path)
{
    FILE *fp;
    RSA *rsa = NULL;

    if ((fp = fopen(rsa_key_path, "rb")) == NULL) {
        return NULL;
    }

    if (!PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL)) {

        fclose(fp);
        return NULL;
    }

    fclose(fp);
    return (rsa);
}

int main()
{
    std::string pathToKey{"privkey.key"};

    std::ifstream f(pathToKey.c_str());
    if (!f.good()) {
        cout << "Invalid path" << endl;
        f.close();
        return 0;
    }

    f.close();

    RSA *privRSA = load_rsa_private_key(pathToKey.c_str());

    if(privRSA == NULL) {
        cout << "Incorrect private key" << endl;
        return 0;
    }

    std::string hash;
    cout << "Enter hash" << endl;
    cin >> hash;

    unsigned char signature[RSA_size(privRSA)];
    unsigned int slen;

    RSA_sign(NID_sha256, (unsigned char *)hash.c_str(), hash.size(), signature, &slen, privRSA);
   // write to file
   RSA_free(privRSA);
   return 0;
}

Can someone explain where is my mistake? I tried different ways to get signature (for example, this one https://eclipsesource.com/blogs/2016/09/07/tutorial-code-signing-and-verification-with-openssl/ )

RSA_sign vs openssl command

0 Answers