String encrypted in Ruby gives: 'BadPaddingException' when decrypted in Java

Viewed 465

I'm using the following Ruby code to encrypt a string

require 'openssl'
require 'base64'

public_key = OpenSSL::PKey::RSA.new(File.read('public_key'))

Base64.encode64(public_key.public_encrypt('Some random string that I want to encrypt.'))

I need to pass the encrypted text in a request to the API, and on the API end, I'm getting the following error(read through the API logs):

javax.crypto.BadPaddingException: Decryption error

Now, I have gone through numerous questions over at Stackoverflow, but given my Ruby code, what I'm doing wrong that the encrypted text can't be decrypted on the API end. Obviously, the API end is using Java.

I'm a completely noob when it comes to encryption, but through encrypting and decrypting in Ruby, I have made sure that the implementation is correct in Ruby. The problem arises when the text encrypted through Ruby language gets decrypted through Java on the API end.

3 Answers

I wrote a very simple Java program to decrypt the output from that very simple Ruby program and it worked fine. Thus, not surprisingly, there's no inherent incompatibility between the ruby openssl module and standard Java cryptography. Without any more info about the Java side, all we can do is list some of the possibilities:

  1. Mismatched keys

The public key you are using must correspond to the private key the Java side is using. If not, you'll likely receive the BadPadding exception.

  1. Formatting issues

Obviously what you transmit, base64 encoded strings with embedded newlines, must be correctly parsed and decoded on the Java side. Getting a single byte wrong can cause the error you see. Most base64 decoders either choke on the newlines and throw an exception or ignore them. Neither explanation would result in a BadPaddingException. Perhaps the Java side is expecting Base64URL instead of Base64? If the Java side is expecting base64url encoding and if it ignores invalid characters then this could be the problem. It's worth trying to remove whitespace including newlines from the base64 output, and if that doesn't work then try encoding with a base64url encoder (again, remove any whitespace from the output).

According to the openJdk source code, BadPaddingException means "if this cipher is in decryption mode, and (un)padding has been requested, but the decrypted data is not bounded by the appropriate padding bytes". This means that there are two separate possible reasons for this exception - whose name is probably misleading, while being technically correct at the lowest level - either the padding is wrong, or the decryption does not match the encryption. In the latter case the result of the decryption would be gibberish, and thus - to interpret the technical language of the source code - would not fit the padding scheme.

From Cryptography book

'PKCS1 padding adds 11 bytes to your original data.'

So you shall encrypt using buffer of size public key length - 11 and use actual public key size for decryption

For 1024 bit RSA keys, the encryption size is normally 117 and decryption buffer size is 128.

Related