I am trying to use Hashicorp Vault to sign a file with RSASSA-PSS-4096. The file is too big for sending it to the server directly, so I want to prehash it locally and then send the digest via POST request to the Vault transit engine.
While the Vault signature verification works, the OpenSSL verification fails. Please see my drafted script:
# Calculate SHA256 hash and convert to base64
sha256sum_base64=$(openssl dgst -sha256 -binary $1 | base64)
# Sign Hash Value with Vault
json_response=$(curl -s \
--header "X-Vault-Token: $(cat token)" \
--request POST \
--data-binary '{"input": "'"$sha256sum_base64"'", "prehashed": true, "signature_algorithm": "pss", "hash_algorithm": "sha2-256"}' \
http://127.0.0.1:8200/v1/transit/sign/rsa_4096)
# Extract base64 signature from the json response.
signature_base64=$(echo $json_response | python3 -c "import sys, json; print(json.load(sys.stdin)['data']['signature'])" | cut -d ":" -f 3)
# Convert signature from base64 to binary and write to file
sigfile=$1__signature.bin
echo $signature_base64 | openssl base64 -d -A -in - -out $sigfile
# Check whether signature is valid via OpenSSL
echo "OpenSSL --> " $(openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:32 -verify rsa_4096_pub.pem -signature $sigfile $1)
# Check whether signature is valid via Vault
signature_vaultformat="vault:v1:$signature_base64"
verify_response=$(curl -s \
--header "X-Vault-Token: $(cat token)" \
--request POST \
--data-binary '{"input": "'"$sha256sum_base64"'", "signature": "'"$signature_vaultformat"'", "prehashed": true, "signature_algorithm": "pss", "hash_algorithm": "sha2-256"}' \
http://127.0.0.1:8200/v1/transit/verify/rsa_4096)
echo "Vault Verify --> " $(echo $verify_response | python3 -c "import sys, json; print(json.load(sys.stdin)['data']['valid'])")
What could be the problem here? I played with rsa_pss_saltlen parameters (e.g. -1) without success. Is there another OpenSSL parameter I am missing? Do I need to consider something for EMSA-PSS?