Problem decrypting DES data generated by des on solaris

I'm having a little bit of trouble here. I've been researching the problem for a little while and I need some help.

The file I am trying to decrypt was encrypted on a solaris box using the following command:

./des -e -k mydeskey [input-file] [output-file]

So I have now tried to write a Java component that will decrypt that file using CipherInputStream.

Here is the code:

String key ="mydeskey";

Cipher cipher = Cipher.getInstance("DES/CBC/NoPadding");

byte[] ivBytes =newbyte[cipher.getBlockSize()];

AlgorithmParameterSpec iv =new IvParameterSpec(ivBytes);

byte[] desKeyData = key.getBytes();

DESKeySpec desKeySpec =new DESKeySpec(desKeyData);

SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");

SecretKey secretKey = keyFactory.generateSecret(desKeySpec);

cipher.init(Cipher.DECRYPT_MODE, secretKey, iv);

InputStream clearInputStream =new CipherInputStream(inputStream, cipher);

byte[] buffer =newbyte[1024];

int bytesRead = 0;

while((bytesRead = clearInputStream.read(buffer)) >= 0){

outputStream.write(buffer, 0, bytesRead);

}

However, the resultant file is not correct. I'm assuming that there is something wrong with the way that I am generating my key. Any help would be greatly appreciated.

[1887 byte] By [cafesofta] at [2007-11-26 17:09:36]
# 1

There was a posting on this topic a few months ago (June 2006 I think) but I can't find it. The problem I remember is the conversion of key string to the 8 byte DES key. I know for certain that just using the getBytes() of the string does not work.

From memory, the only way a match was obtained was by using the -h option to make the -k accept the key as a hex string. I don't think the OP ever found out how the des executable converted the string to the key. If you find the conversion algorithm then please post it.

sabre150a at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 2

I believe the conversion code is the following (I've included all of the variable declarations, etc.):

#define KEYSIZ8

#define KEYSIZB 1024 /* should hit tty line limit first :-) */

char key[KEYSIZB+1];

register int i;

int j,k,l=0;

DES_cblock kk; // here is DES_cblock typedef: typedef unsigned char DES_cblock[8];

for (i=0; i<KEYSIZ; i++)

{

l=0;

k=key[i];

for (j=0; j><8; j++)

{

if (k&1) l++;

k>>=1;

}

if (l & 1)

kk[i]=key[i]&0x7f;

else

kk[i]=key[i]|0x80;

}

Again any help would be appreciated.

Thanks

cafesofta at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 3

> I believe the conversion code is the following (I've

> included all of the variable declarations, etc.):

>

> [code]

> #define KEYSIZ8

> #define KEYSIZB 1024 /* should hit tty line limit

> first :-) */

> char key[KEYSIZB+1];

>

> register int i;

> int j,k,l=0;

> DES_cblock kk; // here is DES_cblock typedef: typedef

> unsigned char DES_cblock[8];

>

> for (i=0; i<KEYSIZ; i++)

> {

>l=0;

> k=key;

>for (j=0; j<8; j++)

> {

> if (k&1) l++;

> k>>=1;

>}

>if (l & 1)

>kk=key&0x7f;

> se

> kk=key|0x80;

> e]

>

I don't think so. Where did you get this from?

sabre150a at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 4

That was an excerpt of code taken from the des.c file located here:

http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/common/openssl/crypto/des/des.c

I encrypt on a solaris box and this is the best guess I have into the code that is actually running on my machine.

Remember that the flags that I use are -d and -k which I believe drops me into the code block I posted.

cafesofta at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 5

You will need to mimic the des_string_to_key() functionality in

http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/common/openssl/crypto/des/str2key.c

It took me about an hour to generate a DES key in Java that matches that generated by des_string_to_key(). At this time I am not able to post the code - I may do so later.

Once you have the des_string_to_key() method working you will need to decrypt using DES/CBC/NoPadding with an IV of 8 zero bytes. Your decrypted data will be padded with zeros.

sabre150a at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 6
If you do get a chance to post the code I am curious to see how it will work.Thanks
cafesofta at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 7
Has anyone else been able to reverse engineer the string to key method for DES? It would really help me out a great deal along with a few others who had similar problems on this forum.Thanks again!
cafesofta at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 8

> Has anyone else been able to reverse engineer the

> string to key method for DES? It would really help me

> out a great deal along with a few others who had

> similar problems on this forum.

I am posting the code. My reluctance to post is due to a problem with DESede, not a problem with DES. The key generation works great with DES (method des_string_to_key() ) and I can encrypt and decrypt in Java and using the 'des' command line utility. When trying to use DESede I have a problem. In CBC mode the first block of any decryption is correct but the rest are not. I know the key is being generated correctly (method des_string_to_2keys()) since I get an exact match between the bytes of the key I generate in Java and the one generated by the 'des' utility.

When I get time I shall try to find where the problem is. The chances are it is something I am doing but I can't find what.

public class LibDes

{

public LibDes() throws Exception

{

cipher_ = Cipher.getInstance("DES/CBC/NoPadding");

}

public byte[] des_string_to_2keys(String keyString) throws Exception

{

final byte[] str = keyString.getBytes();

byte[] key1 = new byte[8];

byte[] key2 = new byte[8];

for (int i = 0; i < str.length; i++)

{

int j = str[i] & 0xff;

if ((i % 32) < 16)

{

if ((i % 16) < 8)

key1[i % 8] ^= (j << 1);

else

key2[i % 8] ^= (j << 1);

}

else

{

j = ((j << 4) & 0xf0) | ((j >>> 4) & 0x0f);

j = ((j << 2) & 0xcc) | ((j >>> 2) & 0x33);

j = ((j << 1) & 0xaa) | ((j >>> 1) & 0x55);

if ((i % 16) < 8)

key1[7-(i % 8)] ^= j;

else

key2[7-(i % 8)] ^= j;

}

}

if (str.length <= 8)

{

for (int index = 0; index < 8; index++)

key2[index] = key1[index];

}

des_set_odd_parity(key1);

key1 = des_cbc_cksum(str, key1, key1);

des_set_odd_parity(key1);

des_set_odd_parity(key2);

key2 = des_cbc_cksum(str, key2, key2);

des_set_odd_parity(key2);

return concat(key1, key2, key1);

}

private static byte[] concat(byte[] a, byte[] b, byte[] c)

{

final byte[] result = new byte[a.length + b.length + c.length];

int k = 0;

for (int index = 0; index < a.length; index++)

result[k++] = a[index];

for (int index = 0; index < b.length; index++)

result[k++] = b[index];

for (int index = 0; index < c.length; index++)

result[k++] = c[index];

return result;

}

/**

*http://web.mit.edu/macdev/Development/MITKerberos/MITKerberosLib/DESLib/Documentation/api.html

*/

public byte[] des_string_to_key(String keyString) throws Exception

{

final byte[] str = keyString.getBytes();

// printHex("Raw", str);

byte[] key = new byte[8];

for (int index = 0; index < str.length; index++)

{

int j = str[index] & 0xff;

if ((index % 16) < 8)

key[index % 8] ^= (j << 1);

else

{

j = ((j << 4) & 0xf0) | ((j >>> 4) & 0x0f);

j = ((j << 2) & 0xcc) | ((j >>> 2) & 0x33);

j = ((j << 1) & 0xaa) | ((j >>> 1) & 0x55);

key[7- (index % 8)] ^= j;

}

}

// printHex("Basic", key);

des_set_odd_parity(key);

// printHex("Parity", key);

// Uses the key as a key in the cbc checksum and

// as the IV, that is why it MUST MUST MUST have

// odd parity.

key = des_cbc_cksum(str, key, key);

//printHex("Checksum", checksum);

// Not really needed by JCE since it ignore the

// parity bit BUT for completeness.

des_set_odd_parity(key);

// printHex("Parity 1", checksum);

return key;

}

public byte[] des_cbc_cksum(byte[] data, byte[] keyBytes, byte[] ivec) throws Exception

{

// Pad the data with zeros until a multiple of 8 bytes

final byte[] paddedData = new byte[((data.length + 7) / 8)*8];

for (int index = 0; index < data.length; index++)

{

paddedData[index] = data[index];

}

// CBC encrypt the key

final SecretKey skey = new SecretKeySpec(keyBytes, "DES");

final IvParameterSpec iv = new IvParameterSpec((byte[])ivec.clone());

cipher_.init(Cipher.ENCRYPT_MODE, skey, iv);

final byte[] rkey = cipher_.doFinal(paddedData);

// The checksum is the last 8 bytes

final byte[] checksum = new byte[8];

for (int index = 0, j = rkey.length - 8; index < 8; index++, j++)

{

checksum[index] = rkey[j];

}

return checksum;

}

public static void des_set_odd_parity(byte[] data)

{

for (int i=0; i<8; i++)

data[i]=(byte)odd_parity[data[i] & 0xff];

}

final Cipher cipher_;

private static final int[] odd_parity =

{

1, 1, 2, 2, 4, 4, 7, 7, 8, 8, 11, 11, 13, 13, 14, 14,

16, 16, 19, 19, 21, 21, 22, 22, 25, 25, 26, 26, 28, 28, 31, 31,

32, 32, 35, 35, 37, 37, 38, 38, 41, 41, 42, 42, 44, 44, 47, 47,

49, 49, 50, 50, 52, 52, 55, 55, 56, 56, 59, 59, 61, 61, 62, 62,

64, 64, 67, 67, 69, 69, 70, 70, 73, 73, 74, 74, 76, 76, 79, 79,

81, 81, 82, 82, 84, 84, 87, 87, 88, 88, 91, 91, 93, 93, 94, 94,

97, 97, 98, 98,100,100,103,103,104,104,107,107,109,109,110,110,

112,112,115,115,117,117,118,118,121,121,122,122,124,124,127,127,

128,128,131,131,133,133,134,134,137,137,138,138,140,140,143,143,

145,145,146,146,148,148,151,151,152,152,155,155,157,157,158,158,

161,161,162,162,164,164,167,167,168,168,171,171,173,173,174,174,

176,176,179,179,181,181,182,182,185,185,186,186,188,188,191,191,

193,193,194,194,196,196,199,199,200,200,203,203,205,205,206,206,

208,208,211,211,213,213,214,214,217,217,218,218,220,220,223,223,

224,224,227,227,229,229,230,230,233,233,234,234,236,236,239,239,

241,241,242,242,244,244,247,247,248,248,251,251,253,253,254,254

};

}

To generate a key you need something along the lines offinal LibDes libdes = new LibDes();

final String keyString = "The quick brown fox jumps over the lazy dog.";

final byte[] keyBytes = libdes.des_string_to_key(keyString);

final byte[] ivBytes = new byte[8];

final IvParameterSpec iv = new IvParameterSpec(ivBytes);

final SecretKey key = new SecretKeySpec(keyBytes, "DES");

final Cipher cipher = Cipher.getInstance("DES/CBC/NoPadding");

cipher.init(Cipher.DECRYPT_MODE, key, iv);

As I said in an earlier post, you will have to deal with the padding yourself.

P.S. This Java code is derived from Eric Young's C source which is distributed under the following license

/* Copyright (C) 1995-1997 Eric Young (eay@mincom.oz.au)

* All rights reserved.

*

* This package is an SSL implementation written

* by Eric Young (eay@mincom.oz.au).

* The implementation was written so as to conform with Netscapes SSL.

*

* This library is free for commercial and non-commercial use as long as

* the following conditions are aheared to. The following conditions

* apply to all code found in this distribution, be it the RC4, RSA,

* lhash, DES, etc., code; not just the SSL code. The SSL documentation

* included with this distribution is covered by the same copyright terms

* except that the holder is Tim Hudson (tjh@mincom.oz.au).

*

* Copyright remains Eric Young's, and as such any Copyright notices in

* the code are not to be removed.

* If this package is used in a product, Eric Young should be given attribution

* as the author of the parts of the library used.

* This can be in the form of a textual message at program startup or

* in documentation (online or textual) provided with the package.

*

* Redistribution and use in source and binary forms, with or without

* modification, are permitted provided that the following conditions

* are met:

* 1. Redistributions of source code must retain the copyright

*notice, this list of conditions and the following disclaimer.

* 2. Redistributions in binary form must reproduce the above copyright

*notice, this list of conditions and the following disclaimer in the

*documentation and/or other materials provided with the distribution.

* 3. All advertising materials mentioning features or use of this software

*must display the following acknowledgement:

*"This product includes cryptographic software written by

*Eric Young (eay@mincom.oz.au)"

*The word 'cryptographic' can be left out if the rouines from the library

*being used are not cryptographic related :-).

* 4. If you include any Windows specific code (or a derivative thereof) from

*the apps directory (application code) you must include an acknowledgement:

*"This product includes software written by Tim Hudson (tjh@mincom.oz.au)"

*

* THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND

* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE

* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE

* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL

* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS

* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)

* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT

* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY

* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF

* SUCH DAMAGE.

*

* The licence and distribution terms for any publically available version or

* derivative of this code cannot be changed. i.e. this code cannot simply be

* copied and put under another distribution licence

* [including the GNU Public Licence.]

*/

I'm not sure how this license effects the use of the Java source posted but if you cannot agree to this license then you should not use this Java code.

Message was edited by:

sabre150

sabre150a at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...
# 9
Thank you very much sabre for all of your help. If I run across the problem I will be sure to post the solution. Thanks again!
cafesofta at 2007-7-8 23:37:26 > top of Java-index,Security,Cryptography...