I have spent several hours on looking for bug in third-party implementation of stream cipher Rabbit. There are several bugs, but one of them:
/**
* @param IV An array of 8 bytes
*/
public void setupIV(final byte[] IV) {
short[] sIV = new short[IV.length>>1];
for(int i=0;i<sIV.length;++i) {
sIV[i] = (short)((IV[i << 1] << 8) | IV[(2 << 1) + 1]);
}
setupIV(sIV);
}
The problem here is that byte IV[i << 1] is casted to int, but since Java doesn't have unsigned types, any value >= 0x80 is casted wrong. Say, byte 0xff becomes 0xffffffff, not 0x000000ff how author of the code above expected.
So I have added simple function:
private int byte2int(byte b){
int ret = b;
if (ret < 0){
ret += 256;
}
return ret;
}
It works, but I'm wondering if this is correct way to do what I need? This solution seems dumb somewhat.