OutputStream: How many bytes are sent?

Dear Friends,

I use the following code to upload a file.

byte[] buf =newbyte[5000];

int nread;

int navailable;

synchronized (in){

while((nread = in.read(buf, 0, buf.length)) >= 0){

//Transfer

out.flush();

out.write(buf, 0, nread);

out.flush();

}

}

buf =null;

But how can I know how many bytes already are sent?

Is there a good way to do that?

Thank you!

With best regards

Inno

[907 byte] By [Innocentusa] at [2007-11-27 0:50:07]
# 1
Count them!count += nread;
sabre150a at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 2

Count them!

count += nread;

Yes, but when I do he has counted the whole filesize in a matter of a few seconds.

The upload is still running.

He is counting too fast!

:(

Here a look at the log:

...

15:49:13: Bytes sent: .....

...

15:49:13: Bytes sent: 490000

15:49:13: Bytes sent: 495000

15:49:13: Bytes sent: 500000

15:49:13: Bytes sent: 505000

15:49:13: Bytes sent: 515000

15:49:13: Bytes sent: 520000

15:49:13: Bytes sent: 525000

15:49:13: Bytes sent: 530000

15:49:13: Bytes sent: 531345

"Finished"

The connection is still running. :(

Inno

Message was edited by:

Innocentus

Innocentusa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 3
How do you know "the connection is still running" ?
sabre150a at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 4

Re:

You would need about 1 Minute for uploading a 2 MB Image.

Whatever size I want to upload:

It magically only needs about 2 or 3 seconds to count from 0 bytes to full filesize.

The other thing is that windows shows the connection as still active.

(Tray-Icon, the two Monitor-Icon, when activated).

The server receives the uploaded images but only after 2 Minutes and not adter 2 seconds.

I think there must be something like a buffer. :(

MfG

Inno

Innocentusa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 5
> I think there must be something like a buffer. :(What is out declared as?Kaj
kajbja at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 6
Declarations:URLConnection connection;.........out = connection.getOutputStream();With best regards!MfGInnoEdit: Does nobody know? :( :( :(
Innocentusa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 7
You need to show more code.
sabre150a at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 8

OK, but don't be scared ;-)

EDIT: Does someone know why?

Do you need more information?

Only the function connect(), pipe() and the declarations are important.

Ignore all the other ones, please. :-)

import java.applet.Applet;

import java.io.OutputStream;

import java.net.URLConnection;

import java.net.URL;

import java.io.IOException;

import java.text.SimpleDateFormat;

import java.util.Date;

import java.util.HashMap;

import java.util.Map;

import java.io.File;

import java.io.InputStream;

import java.util.Random;

import java.io.FileInputStream;

import java.util.Iterator;

import javax.swing.JProgressBar;

/**

*

Title: Client HTTP Request class

*

Description: this class helps to send POST HTTP requests with various form data,

* including files. Cookies can be added to be included in the request.

*

* @author Vlad Patryshev

* @version 1.0

*/

public class ClientHttpRequest extends Thread {

URLConnection connection;

OutputStream os = null;

Map cookies = new HashMap();

long filesize;

private OutputStream osw=null;

protected void connect() throws IOException {

if (os == null)os = connection.getOutputStream();

}

protected void write(char c) throws IOException {

connect();

os.write(c);

}

protected void write(String s) throws IOException {

connect();

os.write(s.getBytes());

}

protected void newline() throws IOException {

connect();

write("\r\n");

}

protected void writeln(String s) throws IOException {

connect();

write(s);

newline();

}

private Random random = new Random();

protected String randomString() {

return Long.toString(random.nextLong(), 36);

}

String boundary = "" + randomString() + randomString() + randomString();

private void boundary() throws IOException {

write("--");

write(boundary);

}

/**

* Creates a new multipart POST HTTP request on a freshly opened URLConnection

*

* @param connection an already open URL connection

* @throws IOException

*/

public ClientHttpRequest(URLConnection connection) throws IOException {

this.connection = connection;

connection.setDoOutput(true);

connection.setRequestProperty("Content-Type",

"multipart/form-data; boundary=" + boundary);

connection.addRequestProperty("Accept-Encoding", "gzip,deflate"); //Bugfix by AS: needed for PHP.

connection.setUseCaches(true);

}

/**

* Creates a new multipart POST HTTP request for a specified URL

*

* @param url the URL to send request to

* @throws IOException

*/

public ClientHttpRequest(URL url) throws IOException {

this(url.openConnection());

}

public ClientHttpRequest() throws IOException {

}

/**

* Creates a new multipart POST HTTP request for a specified URL string

*

* @param urlString the string representation of the URL to send request to

* @throws IOException

*/

public ClientHttpRequest(String urlString) throws IOException {

this(new URL(urlString));

}

private void postCookies() {

StringBuffer cookieList = new StringBuffer();

for (Iterator i = cookies.entrySet().iterator(); i.hasNext();) {

Map.Entry entry = (Map.Entry)(i.next());

cookieList.append(entry.getKey().toString() + "=" + entry.getValue());

if (i.hasNext()) {

cookieList.append("; ");

}

}

if (cookieList.length() > 0) {

connection.setRequestProperty("Cookie", cookieList.toString());

}

}

/**

* adds a cookie to the requst

* @param name cookie name

* @param value cookie value

* @throws IOException

*/

public void setCookie(String name, String value) throws IOException {

cookies.put(name, value);

}

/**

* adds cookies to the request

* @param cookies the cookie "name-to-value" map

* @throws IOException

*/

public void setCookies(Map cookies) throws IOException {

if (cookies == null) return;

this.cookies.putAll(cookies);

}

/**

* adds cookies to the request

* @param cookies array of cookie names and values (cookies[2*i] is a name, cookies[2*i + 1] is a value)

* @throws IOException

*/

public void setCookies(String[] cookies) throws IOException {

if (cookies == null) return;

for (int i = 0; i < cookies.length - 1; i+=2) {

setCookie(cookies[i], cookies[i+1]);

}

}

private void writeName(String name) throws IOException {

newline();

write("Content-Disposition: form-data; name=\"");

write(name);

write('"');

}

/**

* adds a string parameter to the request

* @param name parameter name

* @param value parameter value

* @throws IOException

*/

public void setParameter(String name, String value) throws IOException {

boundary();

writeName(name);

newline(); newline();

writeln(value);

}

private void pipe(InputStream in, OutputStream out) throws IOException {

//System.out.println("Output: "+out);

byte[] buf = new byte[5000];

int nread;

int navailable;

long total = 0; //Menge an Bytes bisher gesendet

int percentage = 0; //Percent done...

int oldpercent = 0;

synchronized (in) {

while((nread = in.read(buf, 0, buf.length)) >= 0) {

//Transfer

out.flush();

out.write(buf, 0, nread);

out.flush();

total += nread; //Wieviel bereits gesendet?

percentage = (int)( ( total * 100.0 ) / filesize );

//System.out.println("STAT_ sent: "+total+" total: "+filesize);

if(oldpercent < percentage){

//System.out.println("%: " + percentage);

SimpleDateFormat sdf = new SimpleDateFormat("HH:mm:ss");

String uhrzeit = sdf.format(new Date());

System.out.println(uhrzeit+": Bytes sent: "+total);

//listener.setProgressStatus(percentage);

}

oldpercent = percentage;

}

}

buf = null;

}

/**

* adds a file parameter to the request

* @param name parameter name

* @param filename the name of the file

* @param is input stream to read the contents of the file from

* @throws IOException

*/

public void setParameter(String name, String filename, InputStream is) throws IOException {

boundary();

writeName(name);

write("; filename=\"");

write(filename);

write('"');

newline();

write("Content-Type: ");

String type = connection.guessContentTypeFromName(filename);

if (type == null) type = "application/octet-stream";

writeln(type);

newline();

pipe(is, os);

newline();

}

/**

* adds a file parameter to the request

* @param name parameter name

* @param file the file to upload

* @throws IOException

*/

public void setParameter(String name, File file) throws IOException {

filesize = file.length();

setParameter(name, file.getPath(), new FileInputStream(file));

}

/**

* adds a parameter to the request; if the parameter is a File, the file is uploaded, otherwise the string value of the parameter is passed in the request

* @param name parameter name

* @param object parameter value, a File or anything else that can be stringified

* @throws IOException

*/

public void setParameter(String name, Object object) throws IOException {

if (object instanceof File) {

setParameter(name, (File) object);

} else {

setParameter(name, object.toString());

}

}

/**

* adds parameters to the request

* @param parameters "name-to-value" map of parameters; if a value is a file, the file is uploaded, otherwise it is stringified and sent in the request

* @throws IOException

*/

public void setParameters(Map parameters) throws IOException {

if (parameters == null) return;

for (Iterator i = parameters.entrySet().iterator(); i.hasNext();) {

Map.Entry entry = (Map.Entry)i.next();

setParameter(entry.getKey().toString(), entry.getValue());

}

}

/**

* adds parameters to the request

* @param parameters array of parameter names and values (parameters[2*i] is a name, parameters[2*i + 1] is a value); if a value is a file, the file is uploaded, otherwise it is stringified and sent in the request

* @throws IOException

*/

public void setParameters(Object[] parameters) throws IOException {

if (parameters == null) return;

for (int i = 0; i < parameters.length - 1; i+=2) {

setParameter(parameters[i].toString(), parameters[i+1]);

}

}

/**

* posts the requests to the server, with all the cookies and parameters that were added

* @return input stream with the server response

* @throws IOException

*/

public InputStream post() throws IOException {

boundary();

writeln("--");

os.close();

return connection.getInputStream();

}

/**

* posts the requests to the server, with all the cookies and parameters that were added before (if any), and with parameters that are passed in the argument

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

*/

public InputStream post(Map parameters) throws IOException {

setParameters(parameters);

return post();

}

/**

* posts the requests to the server, with all the cookies and parameters that were added before (if any), and with parameters that are passed in the argument

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

*/

public InputStream post(Object[] parameters) throws IOException {

setParameters(parameters);

return post();

}

/**

* posts the requests to the server, with all the cookies and parameters that were added before (if any), and with cookies and parameters that are passed in the arguments

* @param cookies request cookies

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

* @see setCookies

*/

public InputStream post(Map cookies, Map parameters) throws IOException {

setCookies(cookies);

setParameters(parameters);

return post();

}

/**

* posts the requests to the server, with all the cookies and parameters that were added before (if any), and with cookies and parameters that are passed in the arguments

* @param cookies request cookies

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

* @see setCookies

*/

public InputStream post(String[] cookies, Object[] parameters) throws IOException {

setCookies(cookies);

setParameters(parameters);

return post();

}

/**

* post the POST request to the server, with the specified parameter

* @param name parameter name

* @param value parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(String name, Object value) throws IOException {

setParameter(name, value);

return post();

}

/**

* post the POST request to the server, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(String name1, Object value1, String name2, Object value2) throws IOException {

setParameter(name1, value1);

return post(name2, value2);

}

/**

* post the POST request to the server, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @param name3 third parameter name

* @param value3 third parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(String name1, Object value1, String name2, Object value2, String name3, Object value3) throws IOException {

setParameter(name1, value1);

return post(name2, value2, name3, value3);

}

/**

* post the POST request to the server, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @param name3 third parameter name

* @param value3 third parameter value

* @param name4 fourth parameter name

* @param value4 fourth parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(String name1, Object value1, String name2, Object value2, String name3, Object value3, String name4, Object value4) throws IOException {

setParameter(name1, value1);

return post(name2, value2, name3, value3, name4, value4);

}

/**

* posts a new request to specified URL, with parameters that are passed in the argument

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

*/

public InputStream post(URL url, Map parameters) throws IOException {

return new ClientHttpRequest(url).post(parameters);

}

/**

* posts a new request to specified URL, with parameters that are passed in the argument

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setParameters

*/

public InputStream post(URL url, Object[] parameters) throws IOException {

return new ClientHttpRequest(url).post(parameters);

}

/**

* posts a new request to specified URL, with cookies and parameters that are passed in the argument

* @param cookies request cookies

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setCookies

* @see setParameters

*/

public InputStream post(URL url, Map cookies, Map parameters) throws IOException {

return new ClientHttpRequest(url).post(cookies, parameters);

}

/**

* posts a new request to specified URL, with cookies and parameters that are passed in the argument

* @param cookies request cookies

* @param parameters request parameters

* @return input stream with the server response

* @throws IOException

* @see setCookies

* @see setParameters

*/

public InputStream post(URL url, String[] cookies, Object[] parameters) throws IOException {

return new ClientHttpRequest(url).post(cookies, parameters);

}

/**

* post the POST request specified URL, with the specified parameter

* @param name parameter name

* @param value parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(URL url, String name1, Object value1) throws IOException {

return new ClientHttpRequest(url).post(name1, value1);

}

/**

* post the POST request to specified URL, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(URL url, String name1, Object value1, String name2, Object value2) throws IOException {

return new ClientHttpRequest(url).post(name1, value1, name2, value2);

}

/**

* post the POST request to specified URL, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @param name3 third parameter name

* @param value3 third parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(URL url, String name1, Object value1, String name2, Object value2, String name3, Object value3) throws IOException {

return new ClientHttpRequest(url).post(name1, value1, name2, value2, name3, value3);

}

/**

* post the POST request to specified URL, with the specified parameters

* @param name1 first parameter name

* @param value1 first parameter value

* @param name2 second parameter name

* @param value2 second parameter value

* @param name3 third parameter name

* @param value3 third parameter value

* @param name4 fourth parameter name

* @param value4 fourth parameter value

* @return input stream with the server response

* @throws IOException

* @see setParameter

*/

public InputStream post(URL url, String name1, Object value1, String name2, Object value2, String name3, Object value3, String name4, Object value4) throws IOException {

return new ClientHttpRequest(url).post(name1, value1, name2, value2, name3, value3, name4, value4);

}

With best regards!

MfG

Inno

Message was edited by:

Innocentus

Innocentusa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 9
> I think there must be something like a buffer. :(You are sending the data across a network using TCP/IP? Then of course there's a buffer. You have no way of telling when the other machine receives a byte that you send.
DrClapa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 10
Well, I have seen many uploaders in java who _can_ show the progress of uploading.How do they do that?Yes, I transfer via TCP/IP.With best regards.Inno
Innocentusa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 11

> Well, I have seen many uploaders in java who _can_

> show the progress of uploading.

> How do they do that?

They do it the same way you do. They count the bytes as they write them to the network.

Are you saying that the server is still receiving the file two minutes after you have finished sending it? You are watching the two machines at the same time in debug mode to determine that? Or is it possible that the server is doing something slow after it receives the file, and it only sends its response after two minutes?

DrClapa at 2007-7-11 23:20:01 > top of Java-index,Java Essentials,Java Programming...
# 12
The server runs fine.But Windows says the connection (LAN) is still running.When I plug my cable out while it is running I get an error :-) .So - it is not possible to count how many bytes already left? :-( :-/With best regards.Inno
Innocentusa at 2007-7-11 23:20:02 > top of Java-index,Java Essentials,Java Programming...
# 13
If you don't buffer the output stream then you can make an estimate of how many bytes have been sent. But of course you can't know how many have been received!
sabre150a at 2007-7-11 23:20:02 > top of Java-index,Java Essentials,Java Programming...
# 14

Well, how can I prevent to buffer or how can I deactivate the buffering-feature?

Will this cause a loss of perfomace or other problems?

(I usually upload Files up to 5 MB).

Does nobody know?

Where can I read ressources about this problem?

With best regards

MfG

Inno

Innocentusa at 2007-7-11 23:20:02 > top of Java-index,Java Essentials,Java Programming...
# 15
The OutputStream of an HttpURLConnection buffers the entire data before it sends it, which it only does when you close it or get the input stream, I forget which. So you can't really accomplish your objective.
ejpa at 2007-7-21 19:53:16 > top of Java-index,Java Essentials,Java Programming...
# 16

Is there an alternative connection-class?

How can I override the methods involved in caching?

I think it is possible because I saw many _normal_(multipart-form-data) java-uploader which shows the status in percent.

With best regards

Inno

URLConnection.setUseCaches(false); doesnt work, btw. there must be more inside of it.

Innocentusa at 2007-7-21 19:53:16 > top of Java-index,Java Essentials,Java Programming...
# 17
You can set a request property to ask for multipart-chunked but I forget the details, sorry ...
ejpa at 2007-7-21 19:53:16 > top of Java-index,Java Essentials,Java Programming...