Problem with FileOutputStream and multithreading
Can anyone explain me why the local variable i
seems to be shared by the different threads in the following code :
import java.io.FileOutputStream;
import java.io.IOException;
publicclass DeadLockimplements Runnable{
publicstaticvoid main(String[] args){
DeadLock d1 =new DeadLock("d1");
DeadLock d2 =new DeadLock("d2");
new Thread(d1).start();
new Thread(d2).start();
}
private String name =null;
public DeadLock(String name){
this.name = name;
}
publicvoid run(){
try{
FileOutputStream fos =new FileOutputStream("c:\\temp\\test.txt",false);
for (int i=0; i < 10; i++){
fos.write((this.name +" : " + i +"\n").getBytes());
}
}catch (IOException ioe){
ioe.printStackTrace();
}catch (Throwable t){
t.printStackTrace();
}
}
}
-- Content of file c:\\temp\\test.txt is something like this --
d1 : 0
d1 : 1
d2 : 2
d2 : 3
d2 : 4
d2 : 5
d1 : 6
d1 : 7
d1 : 8
d1 : 9
[2449 byte] By [
renaud66] at [2007-9-30 19:11:07]
The local variable is not shared (and can't be). You can easily check this by inserting a System.out in your loop. The reason is that each thread opens the same file. You need to open it only once. Here's a version that does what you expect:
import java.io.FileOutputStream;
import java.io.IOException;
public class DeadLock implements Runnable {
FileOutputStream fos = null;
public static void main(String[] args) {
System.out.println("hello ...");
try {
FileOutputStream fos = new FileOutputStream("c:\\temp\\test.txt", false);
DeadLock d1 = new DeadLock("d1",fos);
DeadLock d2 = new DeadLock("d2", fos);
new Thread(d1).start();
new Thread(d2).start();
} catch (Exception e) {
e.printStackTrace();
}
}
private String name = null;
public DeadLock(String name, FileOutputStream fos) {
this.name = name;
this.fos = fos;
}
public void run() {
try {
for (int i = 0; i < 50; i++) {
fos.write((this.name + " : " + i + "\n").getBytes());
fos.flush();
System.out.println(this.name + " : " + i + "\n");
try { Thread.sleep(100); } catch (Exception e) { /* ignore */ }
}
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (Throwable t) {
t.printStackTrace();
}
}
}
Actually I don't think this would be a very good solution at all. Your much better off syncronizing on fos.>>try { Thread.sleep(100); } catch (Exception e) { /* ignore */ }I don't recommend ever writing code like this....
> >>try { Thread.sleep(100); } catch (Exception e) { /* ignore */ }> > I don't recommend ever writing code like this....I don't recommend writting such statements without any arguments against this ...
Thanks for your answers but...
I knew the variable i is not shared among the different threads and I already tried the System.out example. But what I really want to know is how come the output is perfectly ordered :
-- Content of file c:\\temp\\test.txt is something like this --
d1 : 0
d1 : 1
d2 : 2
d2 : 3
d2 : 4
d2 : 5
d1 : 6
d1 : 7
d1 : 8
d1 : 9
Why wouldn't it be something like this :
d1 : 0
d1 : 1
d2 : 0
d1 : 2
d1 : 3
d1 : 4
d2 : 1
d2 : 2
d2 : 3
d2 : 4
d2 : 5
d1 : 5
d1 : 6
d1 : 7
d2 : 6
d2 : 7
d2 : 8
d2 : 9
d1 : 8
d1 : 9
I thought the threads, at one time would desynchronize and some numbers would be skiped or swapped.
so why it is that the 2 loops of the 2 differents threads look like they are perfectly synchronized on the i variable and that thread d1 never overwrites the content of d2 and vice versa.
Thanks
if you run it long enough, you will see that they are not synchronized.
The code below looks still synchronized even with a loop of 0-1000 and Thread.sleep(100) for thread d1 in the middle of the loop
public class DeadLock implements Runnable {
public static void main(String[] args) {
DeadLock d1 = new DeadLock("d1");
DeadLock d2 = new DeadLock("d2");
new Thread(d1).start();
new Thread(d2).start();
}
private String name = null;
public DeadLock(String name) {
this.name = name;
}
public void run() {
try {
FileOutputStream fos = new FileOutputStream("c:\\temp\\test.txt", false);
for (int i = 0; i < 1000; i++) {
if (this.name.equals("d1") && i == 500) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) {
// expected
}
}
fos.write((this.name + " : " + i + "\n").getBytes());
}
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (Throwable t) {
t.printStackTrace();
}
}
}
Looks bizzare to me...
I spoke of the changed code, not the original code...
>>try {>Thread.sleep(100);>} catch (Exception e) { /* ignore */>}What is this supposed to do? (apart from introducing a massive performance bottleneck?)
The only problem with your initial code was that you should have set the FileOutputStream to be appendable. i.e. ("test.txt", true);
If you want to increase the chance of another thread getting cpu time you can call Thread.yield(); Using the following code I believe it generates the result you are looking for.
import java.io.FileOutputStream;
import java.io.IOException;
public class DeadLock implements Runnable {
public static void main(String[] args) {
DeadLock d1 = new DeadLock("d1");
DeadLock d2 = new DeadLock("d2");
new Thread(d1).start();
new Thread(d2).start();
}
private String name = null;
public DeadLock(String name) {
this.name = name;
}
public void run() {
try {
FileOutputStream fos = new FileOutputStream("c:\\test.txt",
true);
for (int i = 0; i < 1000; i++) {
fos.write((this.name + " : " + i + "\n").getBytes());
Thread.yield();
}
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (Throwable t) {
t.printStackTrace();
}
}
}
>
> Actually I don't think this would be a very good
> solution at all. Your much better off syncronizing on
> fos.
>
> >>try { Thread.sleep(100); } catch (Exception e) { /*
> ignore */ }
>
> I don't recommend ever writing code like this....
Errr....
First the code is intended to demonstrate a particular point which has nothing to do with exceptions.
Secondly the generated exception for that code block is often one that you can 'ignore' because it simply doesn't normally occur. And often when it does occur it is because the application is exiting and it going to exit whether you ignore it or not.
Really, the answer is as someone posted: that you're using the same file on multiple threads; and the explanation of your observed behavior is in the FileOutputStream class documentation -- "... depends upon the underlying platform. Some platforms, in particular, allow a file to be opened for writing by only one FileOutputStream (or other file-writing object) at a time" -- it really boils down to "undefined" behavior to try writing to it from multiple threads. Apparently Windows is throwing some writes away -- probably whenever one thread gets to write, if another one tries they go nowhere and then it time-slices back and forth. So synchronize as your desired result dictates.
I'll take this moment to plug attitude I think is valuable: the lang spec says "the programmer who wants to avoid surprises will properly synchronize code" -- people say over and over optimize later and only if needed; these kind of boil down to coding in very obvious language -- be pedantic with your implementation and if the computer still can't outpace your user then STILL DON'T OPTIMIZE IT -- you're probably over-justifying this thing you're writing! I don't want to be around when some space ship drops out of the sky from some bug someone slipped in on an edge-case because the test-case didn't excercise the right area of code -- if you had coded that in obvious, pedantic language, there would be no bug since you'd know what you were talking about -- not what the harness told you. Hopefully that makes sense.
Anyway:
PEACE!
- Steev.
>First the code is intended to demonstrate a particular point which has nothing to do with exceptions.
What is this point? I already asked what the point is a few posts above. It obviously has nothing to do with the original question. Don't post an answer to something when you obviously haven't even read the entire thread, I already asked for clarrification.
>Secondly the generated exception for that code block is often one that you can 'ignore' because it simply doesn't normally occur. And often when it does occur it is because the application is exiting and it going to exit whether you ignore it or not.
There is 2 design patterns for this situation, one invloves using a timer for scheduled jobs and the other using wakeup calls and syncronization for a job queue type situation. I have no problem with catching an InterruptedException with an empty block when it is relevant, that is not what i was concerned about.
If you look at the replies the guy who asked the question started including that sleep code in his reply, like he thought it was the solution to his problem when it obviosly is not. I didn't want him to pick up bad coding habits.
>Some platforms, in particular, allow a file to be opened for writing by only one FileOutputStream (or other file-writing object) at a time" -- it really boils down to "undefined" behavior to try writing to it from multiple threads. Apparently Windows is throwing some writes away -- probably whenever one thread gets to write, if another one tries they go nowhere and then it time-slices back and forth. So synchronize as your desired result dictates.
It's not throwing them away. When outputstreams are created they have start writing at the file start. Either he needs to set one thread to be appender or control the file writing with a static index specifying the next location to write (which would need to be syncronized).
>
> >First the code is intended to demonstrate a
> particular point which has nothing to do with
> exceptions.
>
> What is this point? I already asked what the point
> is a few posts above. It obviously has nothing to do
> with the original question. Don't post an answer to
> something when you obviously haven't even read the
> entire thread, I already asked for clarrification.
I read the entire thread.
But thanks for you concern.
>
> >Secondly the generated exception for that code block
> is often one that you can 'ignore' because it simply
> doesn't normally occur. And often when it does occur
> it is because the application is exiting and it going
> to exit whether you ignore it or not.
>
> There is 2 design patterns for this situation, one
> invloves using a timer for scheduled jobs and the
> other using wakeup calls and syncronization for a job
> queue type situation. I have no problem with catching
> an InterruptedException with an empty block when it is
> relevant, that is not what i was concerned about.
>
First Patterns apply to specific situations. That does not mean that all situations fit a pattern.
Secondly, it is obvious that several people are unclear about what you were "concerned about."
> If you look at the replies the guy who asked the
> question started including that sleep code in his
> reply, like he thought it was the solution to his
> problem when it obviosly is not. I didn't want him to
> pick up bad coding habits.
And the problem would be? The code that I see suggests that the user was just trying to see two threads working at the same time. It had nothing to do with a larger problem.
>
>> >First the code is intended to demonstrate a
>> >particular point which has nothing to do with
>> >exceptions.
>
>> What is this point? I already asked what the point
>> is a few posts above. It obviously has nothing to do
>> with the original question. Don't post an answer to
>> something when you obviously haven't even read the
>> entire thread, I already asked for clarrification.
>I read the entire thread.
>But thanks for you concern.
Yet again you do not answer...What is the point of that code?
>And the problem would be? The code that I see suggests that the user was just trying to see two threads working at the same time.
The problem he was having writing to the file using 2 threads......
Exactly,
my problem is about the output of the FileOutpuStream with concurent thread that is surprinsignly synchronized. Not about exceptions, not about apenders. And I'm not looking for a solution but for an explanation. I wan't to know why in the output file there is exactly 10 lines written by 2 different threads. Each thread writes different number of lines but the total is exactly 10 lines, 'like' if the i variable was shared (but, don't get me wrong, I know it is not shared).
> >
> >> >First the code is intended to demonstrate a
> >> >particular point which has nothing to do with
> >> >exceptions.
> >
> >> What is this point? I already asked what the point
> >> is a few posts above. It obviously has nothing to do
> >> with the original question. Don't post an answer to
> >> something when you obviously haven't even read the
> >> entire thread, I already asked for clarrification.
>
> >I read the entire thread.
> >But thanks for you concern.
>
> Yet again you do not answer...What is the point of
> that code?
And you are missing the point. The point is not whether the original code block was the best solution that one could possibly produce.
The problem was that you took a single line from that block, condemn it, did not specify why it was bad (not to mention a context) and did not provide a better solution.
>
> >And the problem would be? The code that I see
> suggests that the user was just trying to see two
> threads working at the same time.
>
> The problem he was having writing to the file using 2
> threads......
And what does the resolution of that problem have to do with the code snippet that you criticised?
> Exactly,
>
> my problem is about the output of the FileOutpuStream
> with concurent thread that is surprinsignly
> synchronized. Not about exceptions, not about
> apenders. And I'm not looking for a solution but for
> an explanation. I wan't to know why in the output
> file there is exactly 10 lines written by 2 different
> threads. Each thread writes different number of lines
> but the total is exactly 10 lines, 'like' if the i
> variable was shared (but, don't get me wrong, I know
> it is not shared).
You are looking at a tree and ignoring the forest.
Each thread opens the file using append false. So if one thread finishes before the other starts then the second thread will destroy the output of the first.
Opening the same file by two threads is always a bad idea. Access should always by synchronized to a single resourse.
And then file output is buffered which makes it at least possible that two different buffers are being laid down in the same file location.
The results you see have nothing to do with the variable.
> Each thread opens the file using append false. So if
> one thread finishes before the other starts then the
> second thread will destroy the output of the first.
That's right, but how comes only part of the first output is destroyed and part of the second output is destroyed. And surpinsingly the lines of thread d1 that are destroyed are the one that are kept for d2 and the lines of d2 that make their way to the output file are the one that are destroyed from d1.
-- Content of file c:\\temp\\test.txt is something like this --
d1 : 0
d1 : 1
d2 : 2
d2 : 3
d2 : 4
d2 : 5
d1 : 6
d1 : 7
d1 : 8
d1 : 9
You can try it with a longer loop for(int i=0; i < 100000; i++), you'll get the same result.
> > Each thread opens the file using append false. So
> if
> > one thread finishes before the other starts then the
> > second thread will destroy the output of the first.
>
> That's right, but how comes only part of the
> first output is destroyed and part of the
> second output is destroyed.
For my part I don't really care.
As I already said the code is simply wrong in the first place because you are accessing a single resource using two different threads.
If you want to know why that matters then you are going to have to dig deeply into the specifics of the OS. And given that you are on windows that means you had better know C, assembler, the MS debugger, and be willing to learn quite a bit more about how OSes and hardware work.
If however you just want your code to work then you synchronize access to the resource.
Well thank you all for your answers.
I just wanted to know if someone could explain me this strange behavour but if you say it's an OS issue, I will let go. I was wondering if it was a special feature in Java that I never heard of or something else. So as I understand, it's something else
Thanks again.
> Well thank you all for your answers.
> I just wanted to know if someone could explain me this
> strange behavour but if you say it's an OS issue, I
> will let go.
It isn't just an OS issue. It is more of an issue as to how threads and resources work. And the variations that are possible when implementing the conceptual models on real systems.
> I was wondering if it was a special
> feature in Java that I never heard of or something
> else. So as I understand, it's something else
It will apply to other languages as well.
>You can try it with a longer loop for(int i=0; i < 100000; i++), you'll get the same result. Actually i don't get the same result. I get all the entries from d2. Possibly an OS issue?
>>The problem was that you took a single line from that block, condemn it, did not specify why it was bad (not to mention a context) and did not provide a better solution.
My problem was that it did nothing and introduced a perfromance bottleneck like I said. The better solution was to remove the line of course. And to prove you didn't read the thread I DID PROVIDE A SOLUTION.
import java.io.FileOutputStream;import java.io.IOException;public class DeadLock implements Runnable {public static void main(String[] args) {DeadLock d1 = new DeadLock("d1");DeadLock d2 = new DeadLock("d2");new Thread(d1).start();new Thread(d2).start();}private String name = null;public DeadLock(String name) {this.name = name;}public void run() {try {FileOutputStream fos = new FileOutputStream("c:\\test.txt",true);for (int i = 0; i < 1000; i++) {fos.write((this.name + " : " + i + "\n").getBytes());Thread.yield();}} catch (IOException ioe) {ioe.printStackTrace();} catch (Throwable t) {t.printStackTrace();}}}
I'm running on Win2000 pro 5.00.2195 SP4
java.vm.version=1.4.2-b28
Here is the start and the end of the output file for (int i=0;i<100000;i++) :
d1 : 0
d1 : 1
d1 : 2
d1 : 3
d1 : 4
d1 : 5
d1 : 6
d1 : 7
d1 : 8
d1 : 9
d1 : 10
d1 : 11
d1 : 12
d1 : 13
d1 : 14
d1 : 15
d2 : 16
d2 : 17
d2 : 18
d2 : 19
...
d2 : 99979
d2 : 99980
d2 : 99981
d2 : 99982
d2 : 99983
d2 : 99984
d2 : 99985
d2 : 99986
d2 : 99987
d2 : 99988
d2 : 99989
d1 : 99990
d1 : 99991
d1 : 99992
d1 : 99993
d1 : 99994
d1 : 99995
d1 : 99996
d1 : 99997
d1 : 99998
d1 : 99999
So as you can see : exactly 100 000 lines with a perfect mix of d1 and d2.And again I know it's bad programming. I sould not have different threads accessing the same resource but I just wanted to share this strange behavour.
> >>The problem was that you took a single line from
> that block, condemn it, did not specify why it was bad
> (not to mention a context) and did not provide a
> better solution.
>
> My problem was that it did nothing and introduced a
> perfromance bottleneck like I said. The better
> solution was to remove the line of course. And to
> prove you didn't read the thread I DID PROVIDE A
> SOLUTION.
And you provided an explaination above as well.
But this sub-thread is not based on the totality of your posts up until this point in time, but rather on your first post.
And your first post did not provide an explaination nor did it supply an alternative.
