java.io.InvalidClassException when reading serialized object from stream
Hi everyone,
I'd like to serialize an object (it is serializable) and transfer the byte array via url parameter to a servlet (on the same server, same java version,...).
I'm able to serialize it [1] and end up with a byte[], encode the byteArray to a URLEncoded UTF-8 String. This string is passed as an URL parameter to the servlet.
The servlet reads the URL parameter and converts the inputStream back to an object [2]. But this causes an java.io.InvalidClassException [3].
I've already searched the forum for this exception and possible reasons, but most posts deal with a server/client environment where different versions of java are installed. This is _not_ the case here. I also checked the two byte arrays after and before the actual object serialization is performed and these two are _identical_.
The serialized object is of type org.jfree.data.general.AbstractDataset and serializable.
I don't know what I'm missing here, since the two byte[] are identical - so the object _is_ serialized and transmitted to the servlet correctly.
Any pointers? Any help really appreciated.
Best regards,
Kurt
[1] ObjectOutputStream oos = null;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(bos);
oos.writeObject(ds);
oos.flush();
byte[] dsBA = bos.toByteArray();
URLEncoder.encode(new String(dsBA),"UTF-8")
[2] ObjectInputStream ois = null;
Object ds = null;
String dsString = request.getParameter("dsBA");
ByteArrayInputStream bin = new ByteArrayInputStream(dsString.getBytes());
ois = new ObjectInputStream(bin);
try {
ds = ois.readObject();
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
[3] java.io.InvalidClassException: java.util.ArrayList; local class incompatible: stream classdesc serialVersionUID = 8664875232659988799, local class serialVersionUID = 8683452581122892189
>
> [3] java.io.InvalidClassException:
> java.util.ArrayList; local class incompatible: stream
> classdesc serialVersionUID = 8664875232659988799,
> local class serialVersionUID = 8683452581122892189
This means the class version in the serialized form is different from the class version loaded in the JVM where you are deserializing.
This is most likely so because you have a different version of the .class file on both locations. You have to either ensure that both versions are the same, or explictily declare a serialVersionUID in your serializable class.
For more info on the subject and on how to declare this serialVersionUID, see the Java API docs, interface Serializable.
Lokoa at 2007-7-16 23:45:43 >
Hi Loko,
thank you for your answer.
As I mentioned in my initial post, I do the deserialization in a servlet on the same server with the same JVM as the serialization.
The serialization is done in a JSF bean and the servlet is running in the same servlet container than the JSF application.
How is it possible that there are two different versions of a class?
That's would really troubles my mind. I'd understand if there are two different versions of java or a server-client architecture with possible different versions of the dataSource class.
Any more thoughts?
Thank you anyway.
Best Regards,
Kurt
API says
InvalidClassException extends ObjectStreamException
Thrown when the Serialization runtime detects one of the following problems with a Class.
The serial version of the class does not match that of the class descriptor read from the stream
The class contains unknown datatypes
The class does not have an accessible no-arg constructor
Take you pick as to the cause.
Strange. How's that for a thought :)
According to the API docs the serialVersionUID difference is the only way for this exception to occur. So I think we have to assume this is definitely the case.
On the other hand it seems you are right and it should be the same version of ArrayList. I hadn't noticed it was this class, I thought it was a class you wrote. For Serializable classes in the Java API the serialVersionUID is of course not under our control.
So what can we do? Find out on each side which is the serialVersionUID for ArrayList. I took a look in the API docs for the current J2SE 1.5.0 at the Serialized Form for ArrayList, and noticed it documents it to have the same versionUID as the second one in your exception. So the deserializing side is running the current version of Java. Apparently 1.4.2 also documents the same value.
I'm at a loss... Somehow the serialized form contains another versionUID then is documented for either 1.4.2 or 1.5 so it must be an even older version. In your situation I would see if I could get to the serialVersionUID of the sending side using reflection on ArrayList.class. I don't know if you can get to it this way but that's what I'd try.
Lokoa at 2007-7-16 23:45:43 >
> According to the API docs the serialVersionUID difference is the only way for this exception to occurNot according to Sun's API - see my reply #3 and the API, here: http://java.sun.com/j2se/1.5.0/docs/api/index.html
> > According to the API docs the serialVersionUID
> difference is the only way for this exception to
> occur
>
> Not according to Sun's API - see my reply #3 and the
> API, here:
> http://java.sun.com/j2se/1.5.0/docs/api/index.html
Yes but the exception text in the OP's trace slearly shows a difference in serialVersionUID.
Lokoa at 2007-7-16 23:45:43 >
> > > According to the API docs the serialVersionUID
> > difference is the only way for this exception to
> > occur
> >
> > Not according to Sun's API - see my reply #3 and
> the
> > API, here:
> > http://java.sun.com/j2se/1.5.0/docs/api/index.html
>
> Yes but the exception text in the OP's trace slearly
> shows a difference in serialVersionUID.
Yes but your answer was incorrect.
Okay. I should have written "interpreting the API docs for this situation".
Lokoa at 2007-7-16 23:45:43 >
Hi ChuckBing and Loko,
thank you guys for your replies.
I did some debugging related to the serialUID's. First of all, I'm using java 1.4.2 and as said before, there is only one JVM. Anyhow, I checked the serialUID's of the class with the help of ObjectStreamClass for ArrayList and AbstractDataset(the one I'm actually serializing):
ObjectStreamClass osc = ObjectStreamClass.lookup(ArrayList.class);
long serialUID = osc.getSerialVersionUID();
The results are stonning ;)
"Sender" side:
AbstractDataset (long) 1918768939869230744
ArrayList (long) 8683452581122892189
"Receiver" side:
AbstractDataset: (long) 1918768939869230744
ArrayList (long) 8683452581122892189
Conclusion: Both UIDs are identical, which means the classes the sender and receiver JVM are seeing are indeed identical, right?
Since the classes are identical and the byte[] arrays are identical, I have no clue what's going on here!
Sorry to come up with so "bad" news. Wouldn't it be much easier to just find out, there are two different versions of the class and resolve this issue? ;)
I'm happy about any more thoughts and ideas, what I could try.
Thank you guys so much so far.
Have a great day,
Kurt
You're not comparing the correct "things" in your last post above, as can be seen from your original post:stream classdesc serialVersionUID = 8664875232659988799, local class serialVersionUID = 8683452581122892189Your conclusion is incorrect as a result.
So from your debugging info appendix2323, it seems the sender side should be using the same versionUID, confirming that both sides are probably using the same JVM. The only conclusion left is that the sender serializes with the correct UID but somehow in the stream it gets changed... which is unlikely I admit... I don't know how I would proceed, at this moment, apart from trying to follow the entire serialization process itself in a debugger...
However, and don't get your hopes up, but a google on both UIDs reveals you are not alone! Unless this is also you of course :)
http://www.netbeans.org/servlets/ReadMsg?list=nbusers&msgNo=41672
He didn't get any replies but in your place I'd just e-mail him and ask if he ever found out what the problem was.
Keep us posted please, I'm quite intrigued what the cause might be.
Lokoa at 2007-7-16 23:45:43 >
Hi ChuckBings,
I've been comparing the right things to disprove the assumption that the sender and the receiver are seeing different classes, which was an assumption earlier in this thread.
And if I interpret the exception message correctly, the receiver sees an UID of "8664875232659988799" in the serialized stream and want's to deseialize this stream into an ArrayList (with the correct UID).
This leads to two questions: 1) Why does the receiver get's the wrong UID? Shouldn't it be "1918768939869230744" (AbstractDataSet)? and
2) Why does the receiver want's to deserialize into an ArrayList? This should only happen, if it would receive an UID of "8683452581122892189", right?
And propably a third question: Where does the change of the UID in the stream occur? I compared the byte[] arrays and they are identical, so any kind of "transmission error" is excluded here.
But if you think, I'm comparing the wrong things, explain and I'll compare something else for you ;)
Is there a way to find out, which class holds the "unknown" UID of "8664875232659988799"?
Regards,
Kurt
Hi Loko,
didn't I disprove your assumption of changing the UID during transmission (the unlikely case) by comparing the two byte arrays?
They are in fact the identical 602 bytes.
Another mystery to me is, why the receiver wants to deserialize an object with the UID of "8664875232659988799" into an ArrayList, which has indeed an different UID.
Regards and thank you for googling and pointing out the post of Stephen Bixler, I'll email him.
Regards,
Kurt
The conclusion of reply 3 in this post probably applies:
http://forum.java.sun.com/thread.jspa?threadID=630143&messageID=3627417
With what code you've posted, I'm suspicious of the encoding and conversions that you're doing.
Suggestion - simplify the problem, remove everything except what is needed to serialize and then de-serialize an object, and get that working. Then add complexity, one piece at a time.
Hi ChuckBing,
thank you for your post again.
I think I pretty much answered all your questions in my initial post with the code snippets.
I'm using an UTF-8 URL encoded string to transport the serialized object. It has to be a string since it should be an URL parameter within a request to a servlet.
Regarding the post you were refering to in your last post. I do understand the issue of converting byte[] to string, but in my case I've checked the byte[] before it is converted to a string and before the deserialization takes place. The arrays are of identical content.
Is there a better way of transmitting the byte[] via URL parameter to a servlet?
Thank you for your answers,
Kurt
> Hi Loko,
>
> didn't I disprove your assumption of changing the UID
> during transmission (the unlikely case) by comparing
> the two byte arrays?
I suppose you did. Sorry I missed it. To be absolutely clear, you debugged the entire serialization process, starting from an ObjectOutputStream with the correct UID, down to the bytes in the serialized stream, which still contain the correct UID? At receiver side, the correct UID is in the bytes but the deserialization process still thinks there's a different one? And you debugged this as well and still found no point at which it is being changed?
I can't think of any possible explanation for your problem at this moment, given the absolute certainty of there being only one version of ArrayList on your system, and of the correct UID being present in the stream even though at deserialization, a different one is perceived by the serialization runtime.
> They are in fact the identical 602 bytes.
> Another mystery to me is, why the receiver wants to
> deserialize an object with the UID of
> "8664875232659988799" into an ArrayList, which has
> indeed an different UID.
I get the feeling you consider the serialVersionUID a type identifier, used to find out what class the serialized bytes represent.
The serialVersionUID in Serialization is not used to identify the class of the object in the stream. It is used to do a rudimentary form of versioning for different versions of the same class. The idea is that as long as your class changes in a way that is compatible, like just adding a method or adding a field for which you can find a reasonable default value, you keep this UID at the same value. When you make a change that is inherently incompatible, you change the UID.
Wish I could have helped more concretely; regards and good luck!
Lokoa at 2007-7-20 20:20:50 >
> This leads to two questions: 1) Why does the receiver
> get's the wrong UID? Shouldn't it be
> "1918768939869230744" (AbstractDataSet)? and
> 2) Why does the receiver want's to deserialize into
> an ArrayList? This should only happen, if it would
> receive an UID of "8683452581122892189", right?
I'm not familiar with the library you are using, but in all likeliness it is implementing this AbstractDataSet using java.util.ArrayList.
When serializing, all member fields and any Serializable superclass (and its member fields) are serialized. And for each of them, the serialVersionUID checking happens.
Lokoa at 2007-7-20 20:20:50 >
Hi guys,
finally I (with the help of you guys) found the error and so came up with a solution.
First of all, the cause was again human error ;) I trusted my eyeballs, when stating the two byte[] are identical, instead of using java.util.Arrays.equals(byte[] a, byte[] b).
The two byte[] where not identical.
The problem occurs while converting the byte[] to a string and back.
My assumption was that the byte[] to URL encoding is fine by just calling URLEncoder.encode(new String(byte[] a),"UTF-8") since the encoded string is indeed UTF-8 and urlsafe.
The issue is the new String(byte[] a) part of it.
The solution is to make sure, that the byte[] can be converted to a string and back e.g. with org.apache.commons.codec.net.URLCodec.encode(byte[] b).
This will provide you with an byte[] that can be converted to an UTF-8, url encoded string and back to byte[] without flipping a single bit.
I hope this post can help someone in the future running into the same problem.
Best regards and thank you again for your help,
Kurt
Thanks for the solution!!!!!I just met the same problem. ：）
