XMLEncoder / XMLDecoder : problem with foreign characters

XMLEncoder / XMLDecoder work fine with regular English characters, but when I have foreign characters, it gave the following error, what should I do ?

com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 2 of 3-byte UTF-8 sequence.

Continuing ...

java.lang.ArrayIndexOutOfBoundsException: 0

at com.sun.beans.ObjectHandler.dequeueResult(ObjectHandler.java:139)

Thanks.

Frank

[451 byte] By [Ni_Mina] at [2007-11-26 16:21:02]
# 1

Based on what I know of what you are doing (essentially I know nothing): Do something differently.

I guess you're not allowing the parser to determine the encoding of the XML document, but doing it yourself, incorrectly. Probably by allowing the system's default encoding to be used Don't do that.

You could post some code if you want more helpful advice.

DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 2

Alright, you asked for it, you got it, here is the idea and code for it :

I have a license file for my app, I wrote it out using XMLEncoder / XMLDecoder, but since I don't want users to see and change its contents, I want to encrypt it, I searched the web for info and came upon the following article : http://www.javalobby.org/articles/serialization/?source=archives

I don't want to serialize my license class because I might change it later and I still want old programs on the user side with old license class to be able to work with the changed new class. Therefore I used the idea in the above article and write out the file with XMLEncoder, but as a serialized string, that way I can encrypt the string later if I want to, but for now, I am stuck, because the XMLEncoder / XMLDecoder have problem with foreign characters.

Here are my methods :

/** Writes the object and CLOSES the stream. Uses the persistance delegate registered in this class.

* @param os The stream to write to.

* @param o The object to be serialized.

*/

public static void writeXMLObject(OutputStream os,Object o)

{

// Classloader reference must be set since netBeans uses another class loader to loead the bean wich will fail in some circumstances.

ClassLoader oldClassLoader=Thread.currentThread().getContextClassLoader();

Thread.currentThread().setContextClassLoader(Tool_Lib.class.getClassLoader());

XMLEncoder encoder=new XMLEncoder(os);

encoder.setExceptionListener(new ExceptionListener() { public void exceptionThrown(Exception e) { e.printStackTrace(); }});

encoder.writeObject(o);

encoder.close();// Must be closed to write

Thread.currentThread().setContextClassLoader(oldClassLoader);

}

private static ByteArrayOutputStream writeOutputStream=new ByteArrayOutputStream(16384);

/** Writes an object to XML.

* @param out The boject out to write to. Will not be closed.

* @param o The object to write.

*/

public static synchronized void writeAsXML(ObjectOutput out,Object o) throws IOException

{

writeOutputStream.reset();

writeXMLObject(writeOutputStream,o);

String Str_1=writeOutputStream.toString();

out.writeObject(Str_1);

out.flush();

}

public static synchronized void Write_Serialized_XML(String File_Path,Object o) throws IOException { writeAsXML(new ObjectOutputStream(new FileOutputStream(File_Path)),o); }

private static byte[] readBuf=new byte[16384];

/** Reads an object from <code>in</code> using the

* @param in The object input to read from.

* @return The object. Never <code>null</code>.

*/

public static synchronized Object readAsXML(ObjectInput in) throws IOException

{

// Classloader reference must be set since netBeans uses another class loader to load the bean which will fail under some circumstances.

ClassLoader oldClassLoader=Thread.currentThread().getContextClassLoader();

Thread.currentThread().setContextClassLoader(Tool_Lib.class.getClassLoader());

String Str="";

try { Str=(String)in.readObject(); }

catch (Exception e) { e .printStackTrace(); }

XMLDecoder dec=new XMLDecoder(new ByteArrayInputStream(Str.getBytes(),0,Str.length()));

Object o=dec.readObject();

Thread.currentThread().setContextClassLoader(oldClassLoader);

return o;

}

public static synchronized Object Read_Serialized_XML(String File_Path) throws IOException { return readAsXML(new ObjectInputStream(new FileInputStream(File_Path))); }

The top level methods you use are :

void Write_Serialized_XML(String File_Path,Object o)

and

Object Read_Serialized_XML(String File_Path)

You can call them like this :

Write_Serialized_XML("C:/My_App/License",A_License_Object);

A_License_Object = Read_Serialized_XML("C:/My_App/License");

They worked fine for files with regular characters, but had error when the file contains foreign characters.

Thanks for replying !

Frank

Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 3

This jumps out at me:

> I don't want to serialize my license class because I

> might change it later and I still want old programs

> on the user side with old license class to be able to

> work with the changed new class.

Perhaps you should review the documentation on Serializable http://java.sun.com/j2se/1.5.0/docs/api/java/io/Serializable.html

Essentially, if you declare in your codeprivate static final long serialVersionUID = 15;

(or some other number in place of 15), you won't have the issue of incompatible classes, even if you make changes to your class.

Oh, and that is 15L, but for some reason it isn't displaying.

kazea at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 4

> Essentially, if you declare in your code .... you won't have the issue of incompatible classes, even if you make changes to your class.

Really ? Is this true ? To what extent ? Can I add/remove variables, add/remove methods and it will still be able to read back old version of the serialized class ?

Frank

Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 5

Right, it's like I said.XMLDecoder dec =

new XMLDecoder(new ByteArrayInputStream(Str.getBytes(),0,Str.length()));

You are converting the string to bytes using the system's default encoding. And no doubt your XML doesn't have a prolog that declares that it was encoded that way, so your XMLDecoder is following the XML rules and assuming it was encoded in UTF-8.

Actually, reading the API documentation for XMLEncoder, it specifically says it uses UTF-8. So you should use that encoding too, when you convert the string to bytes. You should also use it when you convert the bytes that XMLEncoder produced into that String, but you didn't post the code that does that.

Edit: you're also assuming that the number of bytes produced by Str.getBytes() is going to be equal to the number of chars in Str. That isn't the case either, so you'll need to change that. You might be better off not converting the XML to a string at all, just keeping it in an array of bytes.

Message was edited by:

DrClap

DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 6

Hi DrClap :

Thanks for the pointers, I've changed the methods to look like the following, now it's working and I'm back to square one : the output file is in clear text. I know how to encrypt a string, but not how to encrypt an array of bytes, what can I do to encrypt it before wrting it out ? And then decrypt it before read it in ? Thanks.

public static synchronized void writeAsXML(ObjectOutput out,Object o) throws IOException

{

writeOutputStream.reset();

writeXMLObject(writeOutputStream,o);

byte[] Bt_1=writeOutputStream.toByteArray();

out.writeInt(Bt_1.length);

out.write(Bt_1);

out.flush();

}

public static synchronized Object readAsXML(ObjectInput in) throws IOException

{

// Classloader reference must be set since netBeans uses another class loader to load the bean which will fail under some circumstances.

ClassLoader oldClassLoader=Thread.currentThread().getContextClassLoader();

Thread.currentThread().setContextClassLoader(Tool_Lib.class.getClassLoader());

String Str="";

int length = in.readInt();

if (length > readBuf.length) readBuf = new byte[length];

in.readFully(readBuf,0,length);

XMLDecoder dec=new XMLDecoder(new ByteArrayInputStream(readBuf,0,length));

Object o=dec.readObject();

Thread.currentThread().setContextClassLoader(oldClassLoader);

return o;

}

Frank

Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 7
I have no idea. The encryption I am familiar with works with bytes, not chars, and I'm surprised you have something that works with chars. Normally the result of encryption is not text, that's why I'm surprised.
DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 8
There are some sample programs there : http://www.orlingrabbe.com/javacrypt_index.htmCould you show me the ones your are familiar with ? For instance how to do this :byte[] encrypt(byte[] Input)Frank
Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 9

> There are some sample programs there :

> http://www.orlingrabbe.com/javacrypt_index.htm

I looked at a few of them. Every single one I looked at produces a byte array as the result of encrypting, and the ones that do decrypting also produce a byte array as the result. I don't think that's because I didn't look in the right places. I think that's because all of the useful encryption algorithms work on bytes. I'd suggest you start doing that too.

DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 10
Thanks for the advice, but I just can't find a sample method that works, something like :byte [] encrypt(byte [] plainText,StringBuffer Password) andbyte [] decrypt(byte [] cipherText,StringBuffer Password)Frank
Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 11
I don't know what you're talking about. The class javax.crypto.Cipher has methods that are exactly like that. The page you linked to uses that Cipher class all over the place. How could you not find it?
DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 12

Yes, I looked into them , and used some of the code there, but didn't work, it seems simple enough, but I got error messages, and I looked up the error messages in the forum and found some advices, but they didn't work either, I have these methods :

public byte [] encrypt(byte [] plainText,StringBuffer Password)

{

try

{

IvParameterSpec ivSpec=new IvParameterSpec(i_v);

SecretKey secretKey=new SecretKeySpec(getKeyFromPassword(Password),"AES");

Password=null;

Runtime.getRuntime().gc();

Cipher aes=Cipher.getInstance("AES/CBC/PKCS5Padding");

aes.init(Cipher.ENCRYPT_MODE, secretKey, ivSpec);

byte[] cipherText=aes.doFinal(plainText);

Runtime.getRuntime().gc();

return cipherText;

}

catch (Exception e)

{

e.printStackTrace();

Out("Error in encryption:"+e.getMessage());

}

return null;

}

public byte [] decrypt(byte [] cipherText,StringBuffer Password)

{

try

{

IvParameterSpec ivSpec=new IvParameterSpec(i_v);

SecretKey secretKey=new SecretKeySpec(getKeyFromPassword(Password),"AES");

Password=null;

Runtime.getRuntime().gc();

Cipher aes=Cipher.getInstance("AES/CBC/PKCS5Padding");

aes.init(Cipher.DECRYPT_MODE, secretKey, ivSpec);

byte[] plainText=aes.doFinal(cipherText);

Runtime.getRuntime().gc();

//Out(new String(plainText));

return plainText;

}

catch(Exception e) { e.printStackTrace(); }

return null;

}

//converts key to machine readable form

public byte [] getKeyFromPassword(StringBuffer s) { return md5sum(s.toString().getBytes()); }

public static byte [] md5sum(byte [] buffer)

{

try

{

MessageDigest md5=MessageDigest.getInstance("MD5");

md5.update(buffer);

return md5.digest();

}

catch (NoSuchAlgorithmException e) { Out("AES Algorithm not available on this VM"); }

return null;

}

And I got this error message :

javax.crypto.BadPaddingException: Given final block not properly padded

at com.sun.crypto.provider.SunJCE_f.b(DashoA13*..)

at com.sun.crypto.provider.SunJCE_f.b(DashoA13*..)

at com.sun.crypto.provider.AESCipher.engineDoFinal(DashoA13*..)

at javax.crypto.Cipher.doFinal(DashoA13*..)

So I looked in the forum for an answer, someone suggested that I use no padding, but it didn't work either.

Frank

Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 13
This is now a problem that's nothing to do with XML. I would suggest you post a new question in the Cryptography forum.
DrClapa at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...
# 14
You are right, I just realized and did it, before I sent you the last reply.Thanks.Frank
Ni_Mina at 2007-7-8 22:44:39 > top of Java-index,Java Essentials,Java Programming...