OutputStreamWriter doesn't seem to obey flush()
I'm writing characters to a BufferedWriter whose underlying OutputStream is a TCP socket connection. At the other end of the socket, I see each full buffer of data, but I don't see the last (partial) buffer of data. On the client side, I verify with debug code that I'm writing all the data to the stream, and then I flush the BufferedWriter and all its underlying output streams. But the last partial buffer of data never makes it to the other end of the connection.
here's my client code:
int i = 0;
char[] buf =newchar[1024];
oos =new ObjectOutputStream(tcpSock.getOutputStream());
<some code that reads and writes objects on oos>
out =new BuffereredWriter(new OutputStreamWriter(OOS));
while((i=in.read(buf))!=-1){
out.write(buf, 0, i);
System.err.println("Client: wrote " + i +" chars");
}
out.flush();
oos.flush();
tcpSock.getOutputStream().flush();
server code:
byte[] buf =newbyte[1024];
int i = 0;
long bytesRead = 0;
<verify that fileSize = number of bytes sent by client>
ois =new ObjectInputStream(tcpConnection.getInputStream());
while((i=ois.read(buf))!=-1 && bytesRead + i < fileSize ){
fos.write(buf, 0, i);
bytesRead = bytesRead + i;
System.err.println("Server: bytes read = " + bytesRead);
}
fos.flush();
output shows that only 1024 bytes are read, and the output file only has 1024 characters.
I have a different method on the client that does the same exact thing except it transfers bytes out an OutputStream rather than chars out a BufferedWriter/OutputStreamWriter, and it does not have this problem. The last partial buffer is read by the server in this other method.
# 3
It turns out this failure is occurring under much simpler conditions, with a byte[] to byte[] transfer over an ObjectOutputSream to ObjectInputStream connection. I created a self-contained test program that replicates the problem.
The code below includes a client and server, communicating on port 8090. The client reads a file called "test_in.txt" from the directory where the program is executing and streams the file to the server. The server then streams it to the file "test_out.txt" in the same directory .
I captured the TCP segments and saw that all data from the file was going to the server. But the server code does not read the last partial buffer of data. I believe the problem is with checking for end of file at the server. Since objects are transferred over the same connection before and after the file is streamed, the client sends the file size and the server is supposed to stop reading after all expected bytes are read. There is something wrong with the looping condition I'm using to do this, but I can't figure out why it doesn't work as coded.
This test needs to be run with more than 1024 bytes in the file "test_in.txt", to re-create the problem. You can download the test file I used from http://mysite.verizon.net/midnightjava/test.html. When the file is copied as "test_out.txt", if the last thing you see is not a quote from Forest Gump, the last buffer was not read.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.UnknownHostException;
public class FileCopyTest {
int port = 8090;
InetAddress ip = null;
public static void main (String[] args){
FileCopyTest test = new FileCopyTest();
Server server = test.new Server();
server.start();
Client client = test.new Client();
client.start();
}
class Client extends Thread{
public void run(){
FileInputStream fis = null;
File testFile = new File("test_in.txt");
Long fileSize = new Long(testFile.length());
ObjectOutputStream oos = null;
ObjectInputStream ois = null;
Socket sock = new Socket();
String s = "", s2 = "";
System.out.println("Client: file size = " + fileSize);
try {
fis = new FileInputStream(testFile);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
ip = InetAddress.getByName("localhost");
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
sock = new Socket(ip, port);
oos = new ObjectOutputStream(sock.getOutputStream());
ois = new ObjectInputStream(sock.getInputStream());
oos.writeObject(fileSize);
s = (String) ois.readObject();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
int i = 0;
byte[] buf = new byte[1024];
long bytesWritten = 0;
try {
while((i=fis.read(buf))!=-1) {
oos.write(buf, 0, i);
bytesWritten = bytesWritten + i;
System.err.println("Client: writing " + i + " bytes");
System.err.println("Client: total bytes written: " + bytesWritten);
}
s2 = (String) ois.readObject();
fis.close();
oos.flush();
sock.getOutputStream().flush();
oos.close();
ois.close();
sock.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Client received: " + s + " and " + s2);
}
}
class Server extends Thread{
public void run(){
ObjectOutputStream oos = null;
ObjectInputStream ois = null;
FileOutputStream fos = null;
Long fileSize = null;
try {
ServerSocket server = new ServerSocket(port);
Socket sock = server.accept();
oos = new ObjectOutputStream(sock.getOutputStream());
ois = new ObjectInputStream(sock.getInputStream());
fileSize = (Long) ois.readObject();
oos.writeObject(new String("received file size"));
System.out.println("Server: recevied file size: " + fileSize.longValue());
fos = new FileOutputStream("test_out.txt");
byte[] buf = new byte[1024];
int i = 0;
long bytesRead = 0;
do {
i = ois.read(buf);
fos.write(buf, 0, i);
bytesRead = bytesRead + i;
System.err.println("Server: writing " + i + " bytes");
System.err.println("Server: total bytes written: " + bytesRead);
} while ( bytesRead <= fileSize.longValue());
fos.close();
oos.writeObject(new String("Received file"));
oos.close();
ois.close();
sock.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
