DES/CBC Decrypt of libdes file using javax.crypto
I have a program that receives des encrypted files from a third party. I need to take the file, decrypt it and process it. To do the decryption, I am spawning a program (des.exe) to do the decryption. I would like to rewrite this routine to use the javax.crypto packages and avoid the need of maintaining an external program.
Note that I have no control over the des encryption. I know that it is being done using the des.exe application, or an equivalent, that is based on libdes. I also know that it is using DES and CBC. (I am assuming that it is using PKCS5Padding as it seems the most probable to me.) I know the secret key.
So my code is something like:
String secretDESkey ="12345678";
Cipher myCipher = Cipher.getInstance("DES/CBC/PKCS5Padding");
SecretKeySpec myKeySpec =new SecretKeySpec(secretDESkey.getBytes(),"DES");
IvParameterSpec myIV =new IvParameterSpec(newbyte[8] );
myCipher.init(Cipher.DECRYPT_MODE, myKeySpec, myIV);
InputStream is =new CipherInputStream(new FileInputStream("encrypted.des"), myCipher);
...
is.close();
But of course, this code doesn't exactly work as I would expect... The decrypted content is incorrect.
I think my biggest question is:How can I figure out the initialization vector? I thought that I could read the first 12 bytes of the file and use the 5th to 12th bytes as my initialization vector (see the Payload Format diagram on http://ietfreport.isoc.org/idref/draft-ietf-ipsec-esp-des-cbc/ ), but that does not seem to work. I have also tried all zeros (a blank array), and that does not seem to work.
I must say that I am also surprised that the algorithm is "DES" for SecretKeySpec, but "DES/CBC/PKCS5Padding" for Cipher.getInstance. Should this be consistent? (I gathered this inconsistency from sample code from a forum -- and if I am consistent with DES/CBC/PKCS5Padding, I get an InvalidKeyException saying that DES required.)
Also, am I specifying the secret key OK? I know the key as a string, and the method requires a byte array, so I just use getBytes. Do I need to do something else?
For what it is worth, to decrypt using des.exe, I simply say something like: des -D -k 12345678 encrypted.des
I appreciate any help!
[2595 byte] By [
whitforda] at [2007-10-3 7:05:52]
Without the specification for the des.exe program I suspect you are going to have a big problem. You could spend the next couple of years guessing as to what mode, padding, IV and key are being used. What leads you to expect that des.exe results in the format defined in http://ietfreport.isoc.org/idref/draft-ietf-ipsec-esp-des-cbc/ ? What leads you to think that des.exe converts the -k command line argument to a key by getting the bytes associaed with the value. It is my experience that it is more likely to be a hash of the -k argument that is used for the key.
Is the des.exe based on http://home.planet.nl/~napel/des.htm ? If not, do you have the source code?
Good points! Was kind of hoping that someone was familiar with libdes. I can see that the encryption program is based on libdes 4.01:
des(1) built with libdes v 4.01 - 13-Jan-1997 - eay
I can also see from the export license:
SYMMETRIC 56-BIT DES (DATA ENCRYPTION STANDARD) IN CBC (CIPHER BLOCK CHAINING) MODE WILL BE USED TO ENCRYPT FILES. IN CBC MODE, A MULTIPLE OF 64 BITS ARE ENCIPHERED AT A TIME.
The fact that it says "a multiple of 64 bits are enciphered at a time", would that imply PCKS5Padding? (I am thinking that this implies that NoPadding is not used.)
> Was kind of hoping that someone was
> familiar with libdes.
Your des.exe uses libdes and would seem to be a simple wrapper round libdes.
> I can see that the encryption
> program is based on libdes 4.01:
>
> des(1) built with libdes v 4.01 - 13-Jan-1997 -
> eay
I have just downloaded version 4.01. What you now need is the C code for des.exe. This will tell you how convert it's command line arguments to the arguments reqired by the libdes library.
>
> I can also see from the export license:
>
> SYMMETRIC 56-BIT DES (DATA ENCRYPTION STANDARD) IN
> CBC (CIPHER BLOCK CHAINING) MODE WILL BE USED TO
> ENCRYPT FILES. IN CBC MODE, A MULTIPLE OF 64 BITS ARE
> ENCIPHERED AT A TIME.
DES is defined for a 56 bit key and a block size of 64 bits.
>
> The fact that it says "a multiple of 64 bits are
> enciphered at a time", would that imply PCKS5Padding?
> (I am thinking that this implies that NoPadding is
> not used.)
The padding and IV are the least of your worries. If you specify NoPadding in your Java and get the key right then you will be able to tell from the result what the padding is. If you use zero for the IV on a long enough message (say 5 or more blocks) you should be able to decrypt all but the first block.
Given access to des.exe (and a Windows box) it should not be difficult to work out the padding and possibly the IV but the command line key conversion is a different matter. it could be the simple getBytes() tht you have used but ...
You need to look for the source for des.exe. It is not part of the libdes distribution.
I actually managed to find the source code for des. It is the libdes package (libdes-4.01.tar.gz), there is the des.c file which is the wrapper. My plan is to rebuild the executable (either on a Windows box or a Solaris box -- whichever is easier), and then instrument the code to dump out the key and iv... (I haven't programmed in C in over 8 years, so I am struggling with just getting an environment setup to do a build.)
> I actually managed to find the source code for des.
> It is the libdes package (libdes-4.01.tar.gz), there
> is the des.c file which is the wrapper. My plan is
> to rebuild the executable (either on a Windows box
> or a Solaris box -- whichever is easier), and then
> instrument the code to dump out the key and iv...
> (I haven't programmed in C in over 8 years, so I am
> struggling with just getting an environment setup to
> do a build.)
Yesterday I downloaded libdes-4.01 but it does not have a des.c file. I assume that this is a proprietry C wrapper.
P.S. I managed to build the library just by typing Make but there again I do use Linux!
No des.c? Did you get this:ftp://ftp.psy.uq.oz.au/pub/Crypto/DES/libdes-4.01.tar.gz
> No des.c? Did you get this:
> ftp://ftp.psy.uq.oz.au/pub/Crypto/DES/libdes-4.01.tar.
> gz
No, from ftp://195.113.144.229/MIRRORS/utopia.hacktic.nl/pub/crypto/libraries/des/libdes-l-4.01.tar.gz
If I compre the two distributions, it is obvious that the version I have is based on yours but missing quite a lot.
Unfortunately, the version from ftp.psy.uq.oz.au does not build cleanly!
I managed to build and run 'des' and I can run the attached test program.
The default mode is CBC and the IV is all zeroes. The padding is the problem. I can run the program with NoPadding and get some trailing bytes which do not seem to be PKCS5 yet I can run the program assuming PKCS5 and get no BadPaddingException but I loose data. I suspect there is a BadPaddingException but it is swallowed by the CipherInputStream. If I make the source file an exact multiple of the block length then there is no problem at all if I use NoPadding.
It looks to me like the 'des' program just pads with random bytes but ...
import javax.crypto.*;
import javax.crypto.spec.*;
import java.io.*;
import org.apache.commons.codec.binary.*;
public class DESOptionsTest
{
static class PipeInputStreamToOutputStream implements Runnable
{
PipeInputStreamToOutputStream(InputStream is, OutputStream os)
{
is_ = is;
os_ = os;
}
public void run()
{
try
{
byte[] buffer = new byte[1024];
for(int count = 0; (count = is_.read(buffer)) >= 0;)
{
os_.write(buffer, 0, count);
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
private final InputStream is_;
private final OutputStream os_;
}
public static void main(String[] args)
{
try
{
byte[] ivBytes = new byte[8];
IvParameterSpec iv = new IvParameterSpec(ivBytes);
String hexKey = "0102030405060708";
String plainTextFile = System.getProperty("user.home") + "/user.sql";
String cipherTestFile = plainTextFile + ".enc";
String dir = System.getProperty("user.home") + "/work/dev";
String[] command = {"sh","-c", "des -E -h -k " + hexKey + " " + plainTextFile + " " + cipherTestFile};
final Process process = Runtime.getRuntime().exec(command);
new Thread(new PipeInputStreamToOutputStream(process.getInputStream(), System.out)).start();
new Thread(new PipeInputStreamToOutputStream(process.getErrorStream(), System.err)).start();
int returnCode = process.waitFor();
System.out.println("Return code = " + returnCode);
Hex encoder = new Hex();
byte[] keyBytes = (byte[])encoder.decode(hexKey.toCharArray());
SecretKey key = new SecretKeySpec(keyBytes, "DES");
InputStream istrm = new FileInputStream(cipherTestFile);
Cipher cipher = Cipher.getInstance("DES/CBC/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, iv);
istrm = new BufferedInputStream(istrm);
istrm = new javax.crypto.CipherInputStream(istrm, cipher);
OutputStream ostrm = new FileOutputStream(plainTextFile + ".txt");
byte[] buffer = new byte[1024];
for (int len = 0; (len = istrm.read(buffer)) >= 0;)
{
System.out.write(buffer, 0, len);
ostrm.write(buffer, 0, len);
}
ostrm.close();
istrm.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
Solution to decrypting the 'des' -
import javax.crypto.*;
import javax.crypto.spec.*;
import java.io.*;
import org.apache.commons.codec.binary.*;
import java.util.*;
public class DESOptionsTest_1
{
static class PipeInputStreamToOutputStream implements Runnable
{
PipeInputStreamToOutputStream(InputStream is, OutputStream os)
{
is_ = is;
os_ = os;
}
public void run()
{
try
{
byte[] buffer = new byte[1024];
for(int count = 0; (count = is_.read(buffer)) >= 0;)
{
os_.write(buffer, 0, count);
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
private final InputStream is_;
private final OutputStream os_;
}
public static void main(String[] args) throws Exception
{
// Need to use NoPadding because we deal with the padding
// ourselves.
Cipher cipher = Cipher.getInstance("DES/CBC/NoPadding");
// The IV is an array of the blocksize lenth filled with all zeroes
final byte[] ivBytes = new byte[cipher.getBlockSize()];
final IvParameterSpec iv = new IvParameterSpec(ivBytes);
// The key - in this case specified as 16 hex chars for use with the '-h' parameter'
final String hexKey = "0102030405060708";
String plainText = "The quick brown fox jumps over the lazy dog.12345";
System.out.println("Plain text length = " + plainText.length());
// Encrypt using the 'des' command line program.
// This line will need changing for Windows.
String[] command = {"sh","-c", "echo -n " + plainText + "| des -E -h -k " + hexKey};
final Process process = Runtime.getRuntime().exec(command);
final ByteArrayOutputStream encryptedData = new ByteArrayOutputStream();
new Thread(new PipeInputStreamToOutputStream(process.getInputStream(), encryptedData)).start();
new Thread(new PipeInputStreamToOutputStream(process.getErrorStream(), System.err)).start();
final int returnCode = process.waitFor();
// If no problem then decrypt using the JCE
if (returnCode == 0)
{
final Hex encoder = new Hex();
System.out.println(new String(encoder.encode(encryptedData.toByteArray())));
byte[] keyBytes = (byte[])encoder.decode(hexKey.toCharArray());
SecretKey key = new SecretKeySpec(keyBytes, "DES");
cipher.init(Cipher.DECRYPT_MODE, key, iv);
byte[] decryptedBytes = cipher.doFinal(encryptedData.toByteArray());
// Strip off the padding and convert back to string
// The 'des' program uses a padding scheme where the last byte indicates
// how many bytes of the last block should be used.
String decrypted = new String(Arrays.copyOf(decryptedBytes, decryptedBytes.length - cipher.getBlockSize() + decryptedBytes[decryptedBytes.length-1]), "ascii");
System.out.println("Recovered ... [" + decrypted + "]");
System.out.println(new String(encoder.encode(decryptedBytes)));
}
else
{
System.out.println("Problem encrypting, teturn code = " + returnCode);
}
}
}
This has been very helpful! Thanks a lot! I have the decryption working now.
My last remaining snag is that I don't understand how a key string is translated into a byte array. As long as I specify a hex key and use the equivalent in my code, the decryption works fine.
(The libdes code is very confusing.) The key seems to be in a variable called "key" and it would seem that the following fragment is the hash:
for (i=0; i<KEYSIZ; i++)
{
l=0;
k=key[i];
for (j=0; j><8; j++)
{
if (k&1) l++;
k>>=1;
}
if (l & 1)
kk[i]=key[i]&0x7f;
else
kk[i]=key[i]|0x80;
}
des_set_key((C_Block *)kk,ks);
So I thought that I could do the equivalent in Java with something like:
public static byte [] convertDesKeyStringToByteArray (String strDesKey)
{
assert 8 == strDesKey.length();
byte [] abDesKey = new byte[8];
for (int i = 0; i < strDesKey.length(); i++)
{
int iCh = strDesKey.charAt(i);
int nBits = Integer.bitCount(iCh);
if (1 == (nBits & 0x01))
{
abDesKey[i] = (byte)(iCh & 0x7f);
}
else
{
abDesKey[i] = (byte)(iCh | 0x80);
}
}
return abDesKey;
}
Alas, there is more going on, so I need to trace some more...
> My last remaining snag is that I don't understand how
> a key string is translated into a byte array. As
> long as I specify a hex key and use the equivalent in
> my code, the decryption works fine.
>
> (The libdes code is very confusing.)
Yes! As with most C code, side effects can make it almost impossible to understand.
> The key seems
> to be in a variable called "key" and it would seem
> that the following fragment is the hash:
I did the same as you but I realised that the hashing is a two stage process. I managed to emulate the first stage but failed on the second stage because of it's complex interraction with it's internal key structure. I would have expected that the hashing was just method to convert a string to a key and then use the value as if the -h had been specified but no, it seems to try to take short cuts.
I don't have the incentive to follow the hashing logic but I suspect that you will.
Hi,have you managed, to get things working?I'm wondering, if there is a possibility to convert a key into a hex-key.Regards,Tim
