Reading from a Socket Input Stream

I am trying to use a socket to connect to a FTP site. Everything is fine, except that sometimes, when I try to run multiple RETR commands through the socket to download an entire directory, It most of the time freezes the program. Sometimes it actually will complete. I have a feeling it is a Buffered Reader problem. I am using a Buffered Reader to do the reading and It always seems to get stuck at the same place and I have been changing things for over a week now and i can't figure this out. Here is the code and I will add a comment to where it is getting stuck (it is in the waitForResponse() method. If you have any other comments on how I am trying to write this program I would like to hear them too. Thank you.

import java.io.*;

import java.net.Socket;

import java.util.StringTokenizer;

import java.awt.event.*;

import java.awt.*;

import javax.swing.*;

// This class extends Socket and is designed specifically be used

// for ftp connections only

//***********************************************************************************

publicclass FuriousFTPSocket{

// fields

//*****************************************************************************

private Socket socket =null;

private BufferedReader reader =null;

private BufferedWriter writer =null;

private String log ="";

private BufferedInputStream infoInput =null;

private Timer keepAliveTimer =new Timer(60000,

new ActionListener()

{

publicvoid actionPerformed(ActionEvent e)

{

try

{

sendCommand("NOOP",false);

waitForResponse(false);

}

catch(IOException ioe)

{

System.out.println("Could not keep connection alive.");

ioe.printStackTrace();

}

}

});

// Creates a FuriousFTP Object

//**********************************************************************************

public FuriousFTPSocket(){

}

// Connects to the default port of an FTP server and logs in as

// anonymous/anonymous.

//**********************************************************************************

publicsynchronizedvoid connect(String host,boolean sendToLog)throws IOException{

connect(host, 21, sendToLog);

}

// Connects to an FTP server on the given port and logs in as

// anonymous/anonymous.

//**********************************************************************************

publicsynchronizedvoid connect(String host,int port,boolean sendToLog)throws IOException{

connect(host, port,"anonymous","anonymous", sendToLog);

}

// Connects to an FTP server on the given port and logs in with the supplied

// username and password.

//**********************************************************************************

publicsynchronizedvoid connect(String host,int port, String user,

String pass,boolean sendToLog)throws IOException

{

if (socket !=null)

{

if(sendToLog) addToLog("FuriousFTP is already connected.");

}

else

{

socket =new Socket(host, port);

//infoInput = new BufferedInputStream(socket.getInputStream());

reader =new BufferedReader(new InputStreamReader(socket.getInputStream()));

writer =new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));

String response = waitForResponse(sendToLog);

if (!response.startsWith("220")){

thrownew IOException(

"FuriousFTP received an unknown response when connecting to the FTP server: "

+ response);

}

sendLine("USER " + user, sendToLog);

response = waitForResponse(sendToLog);

if (!response.startsWith("331")){

thrownew IOException(

"FuriousFTP received an unknown response after sending the user: "

+ response);

}

sendLine("PASS " + pass, sendToLog);

response = waitForResponse(sendToLog);

if (!response.startsWith("230")){

thrownew IOException(

"FuriousFTP was unable to log in with the supplied password: "

+ response);

}

socket.setSoTimeout(0);

socket.setKeepAlive(true);

keepAliveTimer.start();

}

}

// Disconnects from the FTP server.

//**********************************************************************************

publicsynchronizedvoid disconnect(boolean sendToLog)throws IOException{

try{

sendLine("QUIT", sendToLog);

waitForResponse(sendToLog);

}

finally{

keepAliveTimer.stop();

socket =null;

}

}

// Returns the working directory of the FTP server it is connected to.

//**********************************************************************************

publicsynchronized String pwd(boolean sendToLog)throws IOException{

sendLine("PWD", sendToLog);

String dir =null;

String response = waitForResponse(sendToLog);

if (response.startsWith("257")){

int firstQuote = response.indexOf('\"');

int secondQuote = response.indexOf('\"', firstQuote + 1);

if (secondQuote > 0){

dir = response.substring(firstQuote + 1, secondQuote);

}

}

return dir;

}

// Changes the working directory. Returns true if successful.

//**********************************************************************************

publicsynchronizedboolean cwd(String dir,boolean sendToLog)throws IOException{

sendLine("CWD " + dir, sendToLog);

String response = waitForResponse(sendToLog);

return (response.startsWith("250"));

}

// Sends a file to be stored on the FTP server. Returns true if the file

// transfer was successful. The file is sent in passive mode to avoid NAT or

// firewall problems at the client end.

//**********************************************************************************

publicsynchronizedboolean stor(File file,boolean sendToLog)throws IOException{

if (file.isDirectory()){

thrownew IOException("FuriousFTP cannot upload a directory.");

}

String filename = file.getName();

return stor(new FileInputStream(file), filename, sendToLog);

}

// Sends a file to be stored on the FTP server. Returns true if the file

// transfer was successful. The file is sent in passive mode to avoid NAT or

// firewall problems at the client end.

//**********************************************************************************

publicsynchronizedboolean stor(InputStream inputStream, String filename,boolean sendToLog)

throws IOException{

BufferedInputStream input =new BufferedInputStream(inputStream);

Socket dataSocket = goPassive(sendToLog);

sendLine("STOR " + filename, sendToLog);

String response = waitForResponse(sendToLog);

if (!response.startsWith ("125") && !response.startsWith("150 ")){

thrownew IOException("FuriousFTP was not allowed to send the file: "

+ response);

}

BufferedOutputStream output =new BufferedOutputStream(dataSocket

.getOutputStream());

byte[] buffer =newbyte[4096];

int bytesRead = 0;

while ((bytesRead = input.read(buffer)) != -1){

output.write(buffer, 0, bytesRead);

}

output.flush();

output.close();

input.close();

response = waitForResponse(sendToLog);

return response.startsWith("226");

}

// Enter binary mode for sending binary files.

//**********************************************************************************

publicsynchronizedboolean bin(boolean sendToLog)throws IOException{

sendLine("TYPE I", sendToLog);

String response = waitForResponse(sendToLog);

return (response.startsWith("200"));

}

// Enter ASCII mode for sending text files. This is usually the default mode.

// Make sure you use binary mode if you are sending images or other binary

// data, as ASCII mode is likely to corrupt them.

//**********************************************************************************

publicsynchronizedboolean ascii(boolean sendToLog)throws IOException{

sendLine("TYPE A", sendToLog);

String response = waitForResponse(sendToLog);

return (response.startsWith("200"));

}

// Sends a raw command to the FTP server.

//**********************************************************************************

privatesynchronizedvoid sendLine(String line,boolean sendToLog)

throws IOException

{

if (socket ==null)

{

addToLog("FuriousFTP is not connected.");

}

else{

try{

writer.write(line +"\r\n");

writer.flush();

if(sendToLog) addToLog("> " + line);

}catch (IOException e)

{

System.out.println("problem while sending command");

e.printStackTrace();

}

}

}

// Reads a line from the Buffered Reader

//*************************************************************

privatesynchronized String readLine(boolean sendToLog)

throws IOException

{

String line = reader.readLine();

//String line = waitForResponse(sendToLog);

if(sendToLog) addToLog("< " + line);

return line;

}

// the method that the console calls to send a raw ftp command

//****************************************************************

publicsynchronizedvoid sendCommand(String command,

boolean sendToLog)

{

if (socket ==null){

if(sendToLog) addToLog("FuriousFTP is not connected.");

}

else{

try

{

sendLine(command, sendToLog);

waitForResponse(sendToLog);

}

catch(IOException e)

{

System.out.println(e.getMessage());

e.printStackTrace();

}

}

}

publicsynchronized Socket goPassive(boolean sendToLog)

throws IOException

{

sendLine("PASV", sendToLog);

String response = waitForResponse(sendToLog);

if (!response.startsWith("227"))

{

System.out.println("error while attempting passive mode");

thrownew IOException("FuriousFTP could not request passive mode: "

+ response);

}

String ip =null;

int port = -1;

int opening = response.indexOf('(');

int closing = response.indexOf(')', opening + 1);

if (closing > 0){

String dataLink = response.substring(opening + 1, closing);

StringTokenizer tokenizer =new StringTokenizer(dataLink,",");

try{

ip = tokenizer.nextToken() +"." + tokenizer.nextToken() +"."

+ tokenizer.nextToken() +"." + tokenizer.nextToken();

port = Integer.parseInt(tokenizer.nextToken()) * 256

+ Integer.parseInt(tokenizer.nextToken());

}catch (Exception e){

System.out.println("bad data link information");

thrownew IOException("FuriousFTP received bad data link information: "

+ response);

}

}

returnnew Socket(ip, port);

}

publicsynchronized String list(boolean sendToLog)

throws IOException

{

if (socket ==null){

if(sendToLog) addToLog("FuriousFTP is not connected.");

returnnull;

}

else{

ascii(sendToLog);

String outputString ="";

Socket dataSocket = goPassive(sendToLog);

BufferedInputStream input =new BufferedInputStream(dataSocket.getInputStream());

sendLine("LIST", sendToLog);

String response = waitForResponse(sendToLog);

if (!response.startsWith ("125") && !response.startsWith("150 ")){

System.out.println("error while getting list");

thrownew IOException("FuriousFTP was not allowed to retrieve the list: "

+ response);

}

ByteArrayOutputStream byteList =new ByteArrayOutputStream();

BufferedOutputStream output =new BufferedOutputStream(byteList);

byte[] buffer =newbyte[4096];

int bytesRead = 0;

while ((bytesRead = input.read(buffer)) != -1){

output.write(buffer, 0, bytesRead);

}

output.flush();

output.close();

input.close();

response = waitForResponse(sendToLog);

outputString = byteList.toString();

if(sendToLog) addToLog(outputString);

return outputString;

}

}

publicsynchronized String nlst(boolean sendToLog)

throws IOException

{

if (socket ==null){

if(sendToLog) addToLog("FuriousFTP is not connected.");

returnnull;

}

else

{

ascii(sendToLog);

String outputString ="";

Socket dataSocket = goPassive(sendToLog);

BufferedInputStream input =new BufferedInputStream(dataSocket.getInputStream());

sendLine("NLST", sendToLog);

String response = waitForResponse(sendToLog);

if (!response.startsWith ("125") && !response.startsWith("150 ")){

System.out.println("error during nlst");

thrownew IOException("FuriousFTP was not allowed to retrieve the list: "

+ response);

}

ByteArrayOutputStream byteList =new ByteArrayOutputStream();

BufferedOutputStream output =new BufferedOutputStream(byteList);

byte[] buffer =newbyte[4096];

int bytesRead = 0;

while ((bytesRead = input.read(buffer)) != -1){

output.write(buffer, 0, bytesRead);

}

output.flush();

output.close();

input.close();

response = waitForResponse(sendToLog);

outputString = byteList.toString();

if(sendToLog) addToLog(outputString);

return outputString;

}

}

publicsynchronized File retr(String fileName,boolean sendToLog)

throws IOException

{

if (socket ==null){

if(sendToLog) addToLog("FuriousFTP is not connected.");

returnnull;

}

else{

bin(sendToLog);

Socket dataSocket = goPassive(sendToLog);

BufferedInputStream input =new BufferedInputStream(dataSocket.getInputStream());

sendLine("RETR " + fileName, sendToLog);

String response = waitForResponse(sendToLog);

if (!response.startsWith("150")){

System.out.println("error during retr");

thrownew IOException("FuriousFTP was not allowed to retrieve the file: "

+ response);

}

ByteArrayOutputStream byteList =new ByteArrayOutputStream();

BufferedOutputStream output =new BufferedOutputStream(byteList);

byte[] buffer =newbyte[4096];

int bytesRead = 0;

while ((bytesRead = input.read(buffer)) != -1){

output.write(buffer, 0, bytesRead);

}

output.flush();

output.close();

input.close();

response = waitForResponse(sendToLog);

if (!response.startsWith("226")){

System.out.println("error after download");

thrownew IOException("FuriousFTP was unable to retrieve the file: "

+ response);

}

File tempFile =new File("/FFTP/Temp/" + fileName);

FileOutputStream fos =new FileOutputStream(tempFile);

fos.write(byteList.toByteArray());

fos.flush();

fos.close();

return tempFile;

}

}

publicvoid clearLog()

{

log ="";

}

// method to wait until the Buffered Reader is ready, then get

// return the first line read as the response string

private String waitForResponse(boolean sendToLog)

throws IOException

{

while(!reader.ready())

{

System.out.println("Waiting for response...");

// THIS IS WHERE IT GETS STUCK!!!

}

String response = readLine(sendToLog);

while(reader.ready())

{

String temp = reader.readLine();

if(sendToLog) addToLog(temp);

}

return response;

}

// returns the log

//************************************************************

public String getLog()

{

return log;

}

// adds one line to the log

//**************************************************************

privatevoid addToLog(String line)

{

log += (line +"\n");

}

}

Message was edited by:

AsSiDuL0Us

[31924 byte] By [AsSiDuL0Usa] at [2007-11-27 1:03:26]
# 1
Seems a lot of code just trying to duplicate the functionality of the Jakarta commons FTP client - http://jakarta.apache.org/commons/net/ . I for one am not willing to spend time going through all that code trying to find an obscure bug.
sabre150a at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 2

I checked out that link and I may end up having to use that if I can't figure this out. I would really like to get this working since I have put so much time into it. Here is exactly the section of code that is the problem. I think I am using BufferedReader incorrectly or something. This method is called after I have written a command to the Socket (which is connected to the FTP server). It works great if I am just doing one command. But I have a loop that sends a RETR command about 52 times in a row. That is when it gets stuck. Most the time it gets stuck after the first few files. I am not that good with streams so this is the best I could come up with. Obviously my implementation is insufficient because it most of the time gets stuck here. But I don't know enough to figure out why. I have tried so many different things. If anyone out there has experience with BufferedReader or reading from a Socket, can you let me know if there is a better way to read from the Socket output stream? The Java tutorial says to use BufferedReader and then the readLine() method so that is why I chose this way. I'll give you all 10 points if you can help me figure this out =). Thank you.

private String waitForResponse(boolean sendToLog)

throws IOException

{

while(!reader.ready())

{

System.out.println("Waiting for response...");

// THIS IS WHERE IT GETS STUCK!!!

}

String response = reader.readLine();

while(reader.ready())

{

String temp = reader.readLine();

if(sendToLog) addToLog(temp);

}

return response;

}

AsSiDuL0Usa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 3

Get rid of the while (!reader.ready()) loop. It is just literally a waste of time. The readLine() will block until some data is available so what is the point of the loop?

And if you're getting stuck in this pointless while() loop, or the readLine() when you take it out, it just means that the sender hasn't sent anything yet.

ejpa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 4

I actually had a problem with the readLine() method as well. But first, the while(!reader.ready())

was to give the server a chance to write its response to the stream before trying to read it. I have tried not using the loop, and still the program freezes. I added that loop because when I call readLine() too many times the program freezes, I guess because it blocks forever. I can't call readLine() just once because there may or may not be multiple lines as a response from the server. But if I don't call readLine() enough, then there may still be another line in the stream and I am using the first line of the response as a way to test for a successful command. Then, when I run the next command, I get that old line in the stream still as what is supposed to be the first line of the new response. Then, the program sees that the response is unexpected (because the response code doesn't match) and throws an exception. That is why I used this for reading every line of the stream: while(reader.ready()) { response += reader.readLine(); }

But this doesn't always read every line of the response, resulting in a line being left in the stream, which gets read on the next command, and again is interpreted as unexpected. I don't know why it does that, its almost like the reader.ready() returns false even though there is a line left in the Stream, or that the full response hasn't yet finished being written to the stream by the server and so reader.ready() returns false. I am so confused and frustrated by this, but I would be really great full to find out why this isn't working or else a better way to read an unknown number of lines from the stream after all the lines have already been written to the stream by the server.

AsSiDuL0Usa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 5

> I actually had a problem with the readLine() method

> as well. But first, the while(!reader.ready())

>

was to give the server a chance to write its

> response to the stream before trying to read it.

Pointless. All you're doing is burning CPU. The readLine() will block until a line arrives, and without burning CPU.

> I have tried not using the loop, and still the program freezes.

Of course, in readLine().

> I added that loop because when I call

> readLine() too many times the program freezes

It will always freeze if you call it once too many and the server still has the socket open. THe only difference you've made is that it's now freezing in your in.ready() loop and smoking the CPU, instead of blocking sanely in readLine().

> Then, when I run the next

> command, I get that old line in the stream still as

> what is supposed to be the first line of the new

> response.

This can only be a programming error. Are you using multiple input streams attached to the socket? If so, don't - attach your BufferedReader once when you acquire the socket and use it everywhere.

> Then, the program sees that the response is

> unexpected (because the response code doesn't match)

> and throws an exception.

Everything that happens from here is due to a programming error.

But it also seems to me that you have a problem in your application protocol. If the server is going to send an arbitrary number of lines and not close the socket it needs to tell you how many lines first. Otherwise this freeze is inevitable.

ejpa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 6

I create the BufferedReader once on successful connection to the FTP server and keep the same one opened until disconnect.

As for the FTP server sending an unknown number of lines, are you saying that I have to know how many lines the FTP server sends back after each type of command? This can be different for each FTP server, so the program wouldn't be generic anymore.

I think if there was a way to have the BufferedReader jump to the end of the stream before I send each command, and then only read one line as a response this might solve my problem of freezing and false exceptions, Something like:

do

{

long charsSkipped = reader.skip(100);

notAtEnd = charsSkipped > 0 ? true : false;

}

while(notAtEnd);

However, even if this were to work, this is just a workaround since I am losing many lines of response. There must be a way to get an arbitrary number of lines from the FTP Server without freezing or missing some of the lines of response.

AsSiDuL0Usa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...
# 7
Your BufferedReader sounds like it should be in its own thread, not a thread which is trying to do other things as well.
ejpa at 2007-7-11 23:38:30 > top of Java-index,Core,Core APIs...