HttpConnection.openInputStream() only gets to 8192 bytes
Hi,
My server wants to send a file bigger than 8k to the cell phone devices. There is javax.microedtion.io.HttpConnection between the cell phone and the server.
So when receiving the file from the server, the code is:
InputStream is = httpConn.openInputStream().
But Java uses a constant 8K in buffer allocation. Thus, it can only successfully receive a file smaller than 8192 bytes. If the server sends a file greater than 8k, the file will be broken in the cell phone side.
Is there a solution how to make the cell phone device receive a file bigger than 8k using JavaME?
[612 byte] By [
freeJeana] at [2007-11-27 10:54:12]
# 1
> Is there a solution how to make the cell phone device
> receive a file bigger than 8k using JavaME?
That should be no problem at all. You simply made a mistake somewhere in your the code. Please show some code!
Message was edited by:
deepspace
# 2
In the servlet in the Server side:
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException {
response.setHeader("Cache-Control", "no-proxy");
response.setContentType("text/plain");
...
PrintWriter out = response.getWriter();
xmlOut.output(root, out)....; // root is a big XML element
out.flush(); // out is a complete and correct XML file at this point
out.close();
}
The server sends the out to to the cell phone device.
In the cell phone side, it handles the XML sending from the server by:
public void run(){
// try catch exception part is ignored here
Connection conn = Connector.open(serverAddress());
httpConn = (httpConnection)conn;
httpConn.setRequestMethod(HttpConnection.POST);
inputStream = httpConn.openInputStream();
// inputStream above is a broken XML (lost data, tags) with size 8K only at this point !
if (httpConn.getResponseCode() == 200){
byte[] buf = new byte[1024];
int read = -1;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while((read = inputStream.read(buf, 0, 1024)) != -1)
baos.write(buf, 0, read);
returnData = new String(baos.toByteArray()).trim();
}
......
}
If root is small enough in the server side, it works well. But if the root is a big XML element, greater than 8K, then some data of the XML element is lost! For example, if the root element in the server side before sending is 16 K. The cell phone side can only receive 8K.
There should be no problem in the server side because at the final point, print out the variable "out" gives the completed correct XML.
The way handling the inputStream in the cell phone side should also be correct. I've tried some other ways to read the inputStream, it gives the same result. After all, when the file is smaller than 8K, it works well.
Could you please help and give some idea?
I've tried to compress the file in the server side. But JavaME doesn't support java.util.zip at all. So the cell phone cannot decompress it.
# 3
Your code looks just fine actually.
Let's test something else:
[code]
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int k = 16*1024;
while(k-->0){
baos.write((int)''a");
}
returnData = baos.toString();
System.out.println("String length":returnData.length());
[code]
What is the result of this code?
I do this because on a lot of devices object size is limited.
# 4
I have run your code. But it seems it never comes out from the loop because k = 16 * 1024 might be too big. However, I've tested with k = 16 and k = 1024.
k = 16, String length: 16664
k = 1024, String length: 17671
k = 16 * 1024, it never comes out. ?
But I don't think the problem is in the part that handles the inputStream from HttpConnection. The problem is that the inputStream itself is wrong. It is not the same as the out in the server side. Because of the Java buffer 8k size, it only contains 8K and corrupts the file. It behaves very badly when the out size is > 8k.
# 5
> I have run your code. But it seems it never comes out
> from the loop because k = 16 * 1024 might be too big.
> However, I've tested with k = 16 and k = 1024.
> > k = 16, String length: 16664
> k = 1024, String length: 17671
> k = 16 * 1024, it never comes out. ?
>
That's strange: with K = 16 string length should also be 16..
> But I don't think the problem is in the part that
> handles the inputStream from HttpConnection. The
> problem is that the inputStream itself is wrong. It
> is not the same as the out in the server side.
> Because of the Java buffer 8k size, it only contains
> 8K and corrupts the file. It behaves very badly when
> the out size is > 8k.
I really don't think so, Try sending the output to System.out and check what it does there. Also check on other devices/emulators...
# 6
Oh, Sorry. It prints out:
k = 16, String length: 16 aaaaaa...
k = 1024, String length: 1024 aaaaaaaaaaaa....
k = 16 * 1024, String length: 16384 . Never comes out...
Before I post this message to the forum, I had used the System.out to check the "out" at the final moment in the server side, right before out.flush(). And it prints the correct and completed XML file.
Then in the cell phone device side, I had also tried many other ways to get the inputStream from the HttpConnection. For example,
http://snippets.dzone.com/posts/show/555
Yet, all of the different ways of handling inputStream give the similar result.
They work when the file size is < 8k.
They don't work when the file size > 8k. Some part of the XML lost!
It's not about how we handle the inputStream. It's about the inputStream has already been corrupted before handling.
At last, I google by "InputStream + 8k", and found some explanation to that. Is that a Java bug?
# 7
> k = 16, String length: 16 aaaaaa...
> k = 1024, String length: 1024 aaaaaaaaaaaa....
> k = 16 * 1024, String length: 16384 . Never comes
Yes, that about right. test k= 8*1024, and test K = 9*1024..
> Before I post this message to the forum, I had used
> the System.out to check the "out" at the final moment
> in the server side, right before out.flush(). And it
> prints the correct and completed XML file.
Server side doesn't count.
> It's not about how we handle the inputStream. It's
> about the inputStream has already been corrupted
> before handling.
You don't know that. until you print it out to System.out.
> At last, I google by "InputStream + 8k", and found
> some explanation to that. Is that a Java bug?
Source?
# 8
I print the "returnData " (in my above code) out and it is the broken version. I've tried several ways to get the "returnData" as I described above. But all of them give similar broken XML.
Do you have other way than the "returnData" to print out the httpConn.openInputStream()?
Some other discussion from google:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4156964
http://forums.alfresco.com/viewtopic.php?p=22230&sid=76d320e5cf5c0e9c5c5870d3e4750afe
http://forum.java.sun.com/thread.jspa?threadID=613505&messageID=3394599
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4700712
# 9
> I print the "returnData " (in my above code) out and
> it is the broken version. I've tried several ways to
> get the "returnData" as I described above. But all of
> them give similar broken XML.
That doesn't count, because something could go wrong with the ByteArray stream.
Just do a simple char by char:
int i;
while( (i = stream.read()!=-1){
System.out.print((char)i);
}
It's slow, but it should work.
> Some other discussion from google:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=415
> 6964
Totally different subject, so not interesting
> http://forums.alfresco.com/viewtopic.php?p=22230&sid=7
> 6d320e5cf5c0e9c5c5870d3e4750afe
I don't see how this has anything to do with your problem.
> http://forum.java.sun.com/thread.jspa?threadID=613505&
> messageID=3394599
He's reading incorrectly
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=470
> 0712
Irrelevant, java SE issue.
Why do you need the whole string at once anyway? Just get out of it what you need while your reading. 't doesn't take as much memory and might even be faster.
# 10
int i;
while( (i = stream.read())!=-1){
System.out.print((char)i);
}
Okay, this piece of code works. So the problem is not in:
httpConnection.openInputStream().
Thus, the problem is in appending strings?
int i = 0;
int length = 0;
byte[] buf = new byte[1024*3];
StringBuffer result = null;
while((i = is.read(buf)) != -1){
String each = new String(buf);
if (result == null){
result = new StringBuffer(each);
}else{
result = result.append(each);
}
buf = new byte[1024*3];
System.out.println(Each: " + each); // OKAY, it prints each block correclty!
length += i;
}
System.out.println("length: " + length);
System.out.println("actual length: " + result.toString().length());
System.out.println("Result: " + result.toString());
Outputs-
length: 16648
actural length: 9216
Result: <broken file> with smaller size than the original file
When I try to append and if the final file size is bigger than some size limit, it will corrupt the file and gives wrong result. Is there any size limit in String/Stringbuffer in JavaME?
# 11
deepspace, are you here?
# 12
As I suspected, you are pusshing the limits of the objects size in J2me. 16K (8192 characters) might be the maxmium.
Try working arround it by interpreting the data on the fly.
