Runtime.exec() is not parsing properly when a argument contains space

Hi,

Runtime.exec("a.exe hai \"how r u\"); to execute a sample application...

In windows the argument received by a.exe are as follows:

hai

how r u (as a single argument)

In Solaris the argument received by a.out are as follows:

hai

"how

r

u"

Why it's happening this way?

using jdk1.5_08

Note: using processbuilder is not possible as of now..

Thanks.

[435 byte] By [chaos_begins_herea] at [2007-11-27 4:46:31]
# 1

It is the shell that interprets escaping so you need

String[] command = {"sh","-c","YourExecutable hai \"how r u\""};

Process p = Runtime.exec(command);

or you can use

String[] command = {"YourExecutable", "hai", "how r u"};

Process p = Runtime.exec(command);

Message was edited by:

sabre150

sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 2
Use the exec(String[]) method instead of the exec(String) method.Edit: Man, am I slow. And (partially) wrong.
masijade.a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 3

> Use the exec(String[]) method instead of the

> exec(String) method.

>

> Edit: Man, am I slow. And (partially) wrong.

No. You are right if you don't try to specify escaping. By specifying the arguments as separate elements of the array, no escaping is required.

sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 4

folks thanks for your reply..

But I think will not solve my problem. These things will surely work, but our customer have this running in the their work place and facing this problem only in solaris and not in windows.. so we can fix this in the next release.. but want could be the solution now !

We need to give a reply to them, saying whether it is our bug or a problem in Java API since it's working fine in Windows.

Question : Why passing whole command with argument as a single works in windows and not in solaris/unix ? Can I conclude and say this is due to a bug in Java API?

Thanks.

Additional Info : when I looked at the source code, they just make use of string tokenizer to convert the single string to array using default separator. so If it works it should in all the 3 platforms or shouldn't work anywhere. But works in Windows.

Message was edited by:

chaos_begins_here

chaos_begins_herea at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 5

> folks thanks for your reply..

>

> But I think will not solve my problem. These things

> will surely work, but our customer have this running

> in the their work place and facing this problem only

> in solaris and not in windows.. so we can fix this in

> the next release.. but want could be the solution now

> !

You could prefix the command with "sh -c " so that the shell is active once again but on some shells this does not work. Must be worth a try.

>

> We need to give a reply to them, saying whether it is

> our bug or a problem in Java API since it's working

> fine in Windows.

Behind the scenes, Windows (or really DOS) has created frigged version of the Unix 'C' exec() calls library so that they behave similar but not the same a the Unix version. I suspect Java just uses this library.

>

> Question : Why passing whole command with argument as

> a single works in windows and not in solaris/unix ?

> Can I conclude and say this is due to a bug in Java

> API?

Definitely NOT.

Edit : I have been racking my brains and I think you need the command string to be

"sh -c \"YourExecutable hai \"how r u\"\""

because the -c option says treat the next argument as a command to the shell.

Message was edited by:

sabre150

sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 6

> "sh -c \"YourExecutable hai \"how r u\"\""

Shouldn't that be

"sh -c \"YourExecutable hai \\\"how r u\\\"\""

or maybe

"sh -c \"YourExecutable hai 'how r u'\""

As single quotes should work just well for the shell as the double quotes.

;-)

masijade.a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 7
After some experimentation I have decided that the approach of using"sh -c \"YourExecutable hai \"how r u\"\""will not work. I know that using a String array with "sh","-c" works but I can't make a single string version work and I can't remember how one fixes it.
sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 8

> > "sh -c \"YourExecutable hai \"how r

> u\"\""

>

> Shouldn't that be

> > "sh -c \"YourExecutable hai \\\"how r u\\\"\""

>

I don't think so but I don't know!

>

> or maybe

>

> > "sh -c \"YourExecutable hai 'how r u'\""

>

>

> As single quotes should work just well for the shell

> as the double quotes.

Single quotes have different meaning when it comes to the expansion of environment variables.

sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 9

>> As single quotes should work just well for the shell

>> as the double quotes.

>

> Single quotes have different meaning when it comes to

> the expansion of environment variables.

True, but if it is only to provide an argument (that does not contain any kind of variable that the shell should interpret) that contains spaces, it would work, and simplify the escaping. (assuming, of course, that you can get the single string version to work at all).

I made that statement on the assumption that shell variables would not be included in the (inner) quoted string.

Then again, you know what they about the word assume.

;-)

masijade.a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 10

thanks for your quick response...

but could you please give me some more info about how to use sh -c

Runtime rt = Runtime.getRuntime();

rt.exec("sh -c \"./a.out\" \"\"ahi\" \"how are you\"\" ");

I tried this..

a.out will write the argument to a file..

I expect the content of the file to be

ahi

how are you

But the file content is ./a.out

Many thanks...

chaos_begins_herea at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 11
> but could you please give me some more info about how> to use sh -cSee reply #7 .
sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 12

Sorry for coming back on this after very long time...

using sh -c doesn't seem to work....

This is what's happening..

we give a input box to the user to type the executable (user may include arguments to that also.. ) , and we simply use Runtime.getRuntime().exec(txt.getValue());

2 options we have:

1. make this work on solaris platform (some how.. need to figure out how !)

2. parse the command string and tokenize into array (doesn't seem to be trivial and it is error prone...)

Thanks.

chaos_begins_herea at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 13

> using sh -c doesn't seem to work....

It does for me so you are using it wrong.

>

> This is what's happening..

> we give a input box to the user to type the

> executable (user may include arguments to that also..

> ) , and we simply use

> Runtime.getRuntime().exec(txt.getValue());

So where is "sh -c" involved here?

sabre150a at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 14

Ok, let me give u more details..

customer are already using our product. To solve this problem we have 2 options..

1. Give some workaround to make it work with existing code.

2. If not possible, rewrite that and give it in a patch..

Snippet from sample code. I print the error stream for more info

Process p = Runtime.getRuntime().exec(args[0]);

p.waitFor();

System.out.println("Exit code : " + p.exitValue());

Trial 1 :

bash-2.05# java Test "sh -c \"./a.out \\\"hello is there any error\\\"\""

sh -c "./a.out \"hello is there any error\""

Exit code : 1

Error Stream :

Input Stream :

bash-2.05# cat data.txt

./a.out

bash-2.05#

Note : content of data.txt should be

./a.out

"hello is there any error

What's wrong on this?

> So where is "sh -c" involved here?

if using sh -c works, then we can give this as an work around for user (ask him to type sh -c with proper escape sequences)

Hope that clarifies..

Thanks...

Note : a.out will write the command line argument to data.txt file.

Message was edited by:

chaos_begins_here

chaos_begins_herea at 2007-7-12 9:59:04 > top of Java-index,Java Essentials,Java Programming...
# 15

> Ok, let me give u more details..

As I said in reply #7, I don't know how to use "sh -c command" as a single string. I do know and understand how to use it as String[] command = {"sh","-c","command"};

You said in one post that you can't modify you code but I suspect that if you don't supply an update to your customer then you will loose him.

sabre150a at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 16

Sorry, I misunderstood your post..

if thought it's possible to use sh -c <cmd> as single command... and that's what I was trying and it's not working.. I will try to run it with array as you said and it is working great !!

So If we have to give it as a patch, then our customer has to wait for 2 months ;(

so giving a workaround would be better.. ok if it is not possible I will work with this...

thanks for your great help...

chaos_begins_herea at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 17
a.out will write the argument to a file..I expect the content of the file to beahihow are youBut the file content is ./a.outYou are aware that in C the argument with index zero is the program name, aren't you?
BIJ001a at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 18

I'm aware of that..

there is a small typo in that...

I invoke

./a.out "hai how" "are"

so i expect the file content to be

./a.out < this is missed out

hai how

are

but file output is just

./a.out (hai how and are missing ... )

I'm writing this with a hope that you may have some solution to my problem ;)

chaos_begins_herea at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 19

> > Ok, let me give u more details..

>

> As I said in reply #7, I don't know how to use "sh -c

> command" as a single string. I do know and understand

> how to use it as String[] command =

> {"sh","-c","command"};

>

exec("sh -c '/bin/ls a b c'");

works, but if you need to escape or quote something in the arg list, it gets hairy.

Edit....

Actually, not that bad.

exec("sh -c '/bin/ls a b \"c d e\"'");

lists

a

b

c d e

Message was edited by:

jverd

jverda at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 20

are you sure that worked for you?

it doesn't work for me...

bash-2.05# sh -c './a.out a b c \"asdf asd\"'

bash-2.05# cat data.txt

./a.out

a

b

c

"asdf

asd"

bash-2.05#

even directly trying to execute it on the shell doesn't work ...

chaos_begins_herea at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 21

> are you sure that worked for you?

>

> it doesn't work for me...

>

> bash-2.05# sh -c './a.out a b c \"asdf asd\"'

> bash-2.05# cat data.txt

> ./a.out

> a

> b

> c

> "asdf

> asd"

> bash-2.05#

>

> even directly trying to execute it on the shell

> doesn't work ...

Look, when you execute it in the Runtime.exec() call, the \" will be interpreted by Java to be a simple quote. Your

sh -c './a.out a b c \"asdf asd\"'

will become

sh -c './a.out a b c "asdf asd"'

once interpreted and passed to the shell. Try running that one.

kevjavaa at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 22

ok, sorry for that.. I should n't have given \" when executing in shell.. really sorry..

but when I do the same from runtime.exec().. it doesn't work...

watch me..

Runtime.getRuntime().exec("sh -c './a.out a b c \"x y z\" ' ");

argument to shell will be './a.out a b c "x y z" ' <-- right?

but still I couldn't get the output I want..

chaos_begins_herea at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 23

> ok, sorry for that.. I should n't have given \" when

> executing in shell.. really sorry..

>

> but when I do the same from runtime.exec().. it

> doesn't work...

>

> watch me..

> > Runtime.getRuntime().exec("sh -c './a.out a b c \"x y

> z\" ' ");

>

>

> argument to shell will be './a.out a b c "x y z" '

No, it will be

./a.out a b c "x y z"

without the 'single quotes' around the whole thing.

> but still I couldn't get the output I want..

What did you get?

The exact command I ran was Runtime.getRuntime().exec("C:/cygwin/bin/sh -c 'ls -l /tmp / > \"/tmp/zzz www\"'");

and it did exactly what I'd expect, the same as

ls -l /tmp / > "/tmp/zzz www"

in a terminal window. It put the results of the ls in a file in /tmp called zzz www.

jverda at 2007-7-21 21:13:01 > top of Java-index,Java Essentials,Java Programming...
# 24

> The exact command I ran was

> Runtime.getRuntime().exec("C:/cygwin/bin/sh -c

> 'ls -l /tmp / > \"/tmp/zzz www\"'");

Using String command = "sh -c 'ls -l /tmp / > \"/tmp/zzz www\"'";

final Process process = Runtime.getRuntime().exec(command);

under Linux FC5 JDK5 and JDK6 gives the error message

-l: -c: line 0: unexpected EOF while looking for matching `''

-l: -c: line 1: syntax error: unexpected end of file

Return code = 2

which is why I say I can't get the quoting to work when using a single string (as opposed to a string array) for the command. And YES, I have checked the quotes very carefully.

This would suggest that Java Runtime.exec() has a problem, FC5 has a problem or (most likely) that there is something that I am missing.

Message was edited by:

sabre150

sabre150a at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 25
(deleted)Message was edited by: BIJ001
BIJ001a at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 26
Sabre, it doesn't work for me under 1.4 on a Mac, but I haven't bothered to capture stderr to see exactly what's wrong. Works fine under 1.5 on cygwin though.
jverda at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 27
I added the stderr gobbler and got zsh:1: unmatched 'Works fine in a terminal though.
jverda at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 28

Can we file a bug for this?

if it is not possible to make it work a single string, I will have to take array approach..

now I have this question:

1. what will have if the machine doesn't have bourne shell (assuming I use sh -c command to execute)

2. How to find the default shell in solaris? If something like a system variable which will give me the default shell, so that I can use it to execute the script be sure that it exist in the mahine..

Thanks to all of you for all your help...

chaos_begins_herea at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 29

> Can we file a bug for this?

I don't think so! I think it is most probably a feature of the underlying 'C' exec() call.

>

> if it is not possible to make it work a single

> string, I will have to take array approach..

In my view the best solution. The user should be able to specify ALL the elements of the array, not just the command to be executes with the shell. You could easily create a simple language that would allow you to specify the array elements. Regular expressions are your friend.

>

> now I have this question:

>

> 1. what will have if the machine doesn't have bourne

> shell (assuming I use sh -c command to execute)

bsh, bash, csh and tcsh ALL have the -c option. I'm not sure about ksh. It is normal for /bin/sh to be linked to one of the shells.

> 2. How to find the default shell in solaris? If

> something like a system variable which will give me

> the default shell, so that I can use it to execute

> the script be sure that it exist in the mahine..

When one adds a user one specifies which shell he should use.

Instead of using String[] command = {"sh","-c","......."}; you can explicitly use a particular shell. For example String[] command = {"csh","-c","......."}; . If I had to allow my users to specify a command for Runtime.exec() I would create a syntax that would let the user define which shell is being used and the option that defines the script location. I consider it a VERY VERY dangerous approach and would try to find some way round using it.

sabre150a at 2007-7-21 21:13:02 > top of Java-index,Java Essentials,Java Programming...
# 30

> I would create a syntax that would let the user define which shell is being> used and the option that defines the script location

Seriously I too thought to give a option like this which lets the user specify the shell to use. That's really a nice thing to have.

Many Thanks !!

chaos_begins_herea at 2007-7-21 21:13:06 > top of Java-index,Java Essentials,Java Programming...