SSL HandShake problem in Linux

Hello

I have developed an application that opens a SSLSocket to connect to a remote server. It works on windows but not on Linux machines.

Running the application on Linux machines (RH 7.3, RHEL4.3 and CentOS), I find the following exception when executing "socket.startHandshake()":

java.net.SocketException: Socket closed

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at com.sun.net.ssl.internal.ssl.InputRecord.a(DashoA6275)

at com.sun.net.ssl.internal.ssl.InputRecord.a(DashoA6275)

at com.sun.net.ssl.internal.ssl.InputRecord.read(DashoA6275)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.a(DashoA6275)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.j(DashoA6275)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(DashoA6275)

If I run the application using -Djavax.net.debug=ssl, I find that I am being disconnected after the following:

TSY_AUTH_CONNECTION, WRITE: TLSv1 Handshake, length = 59

TSY_AUTH_CONNECTION, WRITE: SSLv2 client hello message, length = 77

TSY_AUTH_CONNECTION, READ: TLSv1 Handshake, length = 74

*** ServerHello, TLSv1

RandomCookie: GMT: 1154497186 bytes = { 76, 79, 199, 144, 134, 187, 14, 169, 211, 237, 53, 213, 7, 81, 172, 229, 95, 59, 255, 118, 99, 39, 108, 11, 92, 17, 104, 251 }

Session ID: {109, 35, 213, 87, 96, 219, 239, 108, 37, 232, 24, 254, 136, 87, 185, 85, 39, 231, 204, 106, 90, 210, 177, 108, 25, 94, 93, 77, 26, 100, 208, 180}

Cipher Suite: SSL_RSA_WITH_RC4_128_SHA

Compression Method: 0

***

%% Created: [Session-1, SSL_RSA_WITH_RC4_128_SHA]

** SSL_RSA_WITH_RC4_128_SHA

However, If I do the same on a Windows XP, the connection is established correctly:

TSY_AUTH_CONNECTION, WRITE: TLSv1 Handshake, length = 73

TSY_AUTH_CONNECTION, WRITE: SSLv2 client hello message, length = 98

TSY_AUTH_CONNECTION, READ: TLSv1 Handshake, length = 74

*** ServerHello, TLSv1

RandomCookie: GMT: 1154450754 bytes = { 69, 48, 1, 8, 180, 91, 219, 200, 63, 169, 220, 19, 135, 6, 139, 199, 230, 97, 197, 137, 133, 66, 151, 98, 185, 113, 49, 214 }

Session ID: {133, 29, 72, 210, 35, 193, 122, 24, 247, 221, 37, 71, 15, 127, 81, 168, 189, 202, 211, 46, 85, 237, 187, 118, 145, 173, 205, 14, 141, 156, 149, 177}

Cipher Suite: SSL_RSA_WITH_RC4_128_SHA

Compression Method: 0

***

%% Created: [Session-1, SSL_RSA_WITH_RC4_128_SHA]

** SSL_RSA_WITH_RC4_128_SHA

TSY_AUTH_CONNECTION, READ: TLSv1 Handshake, length = 2072

*** Certificate chainchain [0] = [.........................

Could anybody please give me a hand with this issue? I am totally lost.

Thanks in advance

[2805 byte] By [jjmarmola] at [2007-11-26 18:11:17]
# 1
Can you get an SSL trace of any kind from the remote server?And can you please stop cross-posting this question. The correct forum for it is right here.
ejpa at 2007-7-9 5:43:47 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 2

I tried, but I can't. The remote server belongs to an external company that provides no support to us.

Sorry about that. I first asked in the Java Programming forum and there I was told to ask in one of the Security forums or the Sockets forum. And did it in the Security forum and in the Sockets forum. It won't happen again.

Thank you.

jjmarmola at 2007-7-9 5:43:47 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 3

OK, so the Linux client is closing its own connection before the server can send its certificate. Let me look into this a bit more.

Incidentally you don't need to call startHandshake(), it will happen automatically with the first I/O to the socket. Or is this a re-handshake? If not, can you try it without the startHandshake call?

Or do you by any chance use Thread.interrupt() in this application? That can close sockets in Linux but not in Windows.

ejpa at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 4

The client application sends the ClientHello and receives the ServerHello. Then, according to the TLS handshake protocol for client/server authorization using RSA, the client should receive the server certificate, but it doesn't. And after a few seconds the socket is closed by the server. I suppose the server has a timeout.

This is not a re-handshake. I tried without the startHandshake() and it happened automatically, as you said, but the problem happened again.

No Thread.interrupt() is used.

Thank you.

jjmarmola at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 5
Needless to say that the network conditions are the same in both cases (testing on a Windows and on a Linux): the network is the same and I use the same proxy.
jjmarmola at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 6
> And after a few seconds> the socket is closed by the server.No. The socket is being closed by the client for some reason. A close by the server would look entirely different. Is there possibly an asynchronous close somewhere else in your code?
ejpa at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 7

I have checked and I don't find any asynchronous close.

I realized that the Cipher Suites indicated in the client Hello are different. In Linux the following cipher suites do not appear:

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_DSS_WITH_AES_128_CBC_SHA

But the keystore is the same in both, Windows and Linux because I copied in Linux the cacerts used in Windows.

So, if I use the following code to enable the cipher suites, it also works in Windows but not in Linux. In Linux it says that it can't find the mentioned cipher suites.

String[] cipherSuites = {"SSL_RSA_WITH_RC4_128_MD5",

"SSL_RSA_WITH_RC4_128_SHA",

"TLS_RSA_WITH_AES_128_CBC_SHA",

"TLS_DHE_RSA_WITH_AES_128_CBC_SHA", "TLS_DHE_DSS_WITH_AES_128_CBC_SHA","SSL_RSA_WITH_3DES_EDE_CBC_SHA","SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA",

"SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA",

"SSL_RSA_WITH_DES_CBC_SHA",

"SSL_DHE_RSA_WITH_DES_CBC_SHA",

"SSL_DHE_DSS_WITH_DES_CBC_SHA",

"SSL_RSA_EXPORT_WITH_RC4_40_MD5",

"SSL_RSA_EXPORT_WITH_DES40_CBC_SHA",

"SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA","SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA"};

tsySocket = (SSLSocket)sf.createSocket(address, port);tsySocket.setEnabledCipherSuites(cipherSuites);

Could this be the problem? How can I add cipher suites?

Thank you

jjmarmola at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 8
The cipher suites aren't the problem because the server and client have succeeded in negotiating the session, using SSL_RSA_WITH_RC4_128_SHA.But does the Linux client have a truststore that will trust a cert with RSA keys?
ejpa at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 9

I think that the truststore does trust a cert with RSA keys. I use the same in both (Linux and Windows).

In Linux, if I use -Djavax.net.debug=all, once the session is created I see the following:

%% Created: [Session-1, SSL_RSA_WITH_RC4_128_SHA]

** SSL_RSA_WITH_RC4_128_SHA

[read] MD5 and SHA1 hashes: len = 74

0000: 02 00 00 46 03 01 45 D5AA 44 68 02 76 87 D9 0E ...F..E..Dh.v...

0010: 53 A0 DB 97 CF 56 11 CAE3 8B 5F 30 44 26 2E 6F S....V...._0D&.o

0020: CD 8C A9 DD 31 B4 20 6FED F5 9F 91 C0 AD 97 68 ....1. o.......h

0030: BE FD 59 62 C6 B5 86 301D 39 90 03 06 09 7C E3 ..Yb...0.9......

0040: DA 6C A8 10 FE C3 B7 0005 00.l........

[Raw read]: length = 5

0000: 16 03 01 08 18 .....

[Raw read]: length = 1376

0000: 0B 00 08 14 00 08 11 0004 84 30 82 04 80 30 82 ..........0...0.

0010: 03 E9 A0 03 02 01 02 0210 62 76 EC A2 A8 88 DA .........bv.....

0020: 1A 17 11 83 6B F0 1D A922 30 0D 06 09 2A 86 48 ....k..."0...*.H

...........lots of bytes till the end of the message..........

0540: 30 81 BA 31 1F 30 1D 0603 55 04 0A 13 16 56 65 0..1.0...U....Ve

0550: 72 69 53 69 67 6E 20 5472 75 73 74 20 4E 65 74 riSign Trust Net

TsyAuthSocketReader, handling exception: java.net.SocketException: Connection re

set

TsyAuthSocketReader, SEND TLSv1 ALERT: fatal, description = unexpected_message

TsyAuthSocketReader, WRITE: TLSv1 Alert, length = 2

TsyAuthSocketReader, Exception sending alert: java.net.SocketException: Broken p

ipe

TsyAuthSocketReader, called closeSocket()

In Windows I can see:

%% Created: [Session-1, SSL_RSA_WITH_RC4_128_SHA]

** SSL_RSA_WITH_RC4_128_SHA

[read] MD5 and SHA1 hashes: len = 74

0000: 02 00 00 46 03 01 45 D583 48 2E 1D 42 DB E2 F9 ...F..E..H..B...

0010: F9 26 17 0E F9 F9 CB D5C6 B2 F2 1F 72 64 33 B3 .&..........rd3.

0020: BD FD 0D 30 C6 5B 20 3B26 A8 64 9A 43 A3 DF CB ...0.[ ;&.d.C...

0030: 3B 5C 18 94 11 A5 89 7C1A F6 B3 CD D7 77 6A 2B ;\...........wj+

0040: 36 E5 46 71 9F E4 6F 0005 006.Fq..o...

TsyAuthSocketReader, READ: TLSv1 Handshake, length = 2072

*** Certificate chain

chain [0] = [

............

I don't know but to me it seems that the Linux client is not receiving the whole Server Certificate. In Windows the message is 2072 bytes, but in Windows it is 1376!

Using the Ethereal in Windows, I see that the client receives two messages:

- Server Hello Certificate [Unreassembled packet], which is 1380 bytes long.

- Continuation Data [Unreassembled packet], which is 705 bytes long.

But in Linux the client receives just a message that is 5 bytes long and then another one 1376 bytes long. So to me, it seems that for some reason the Server Hello Certificate is cut.

jjmarmola at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...
# 10

> TsyAuthSocketReader, handling exception: java.net.SocketException: Connection reset

Didn't see that before. The server is resetting the connection for some reason.

Is it the same server in both the Windows and the Linux cases?

Do you do any playing around with socket options in your client code?

Does the server maybe have some configuration or permission which prevents it talking to the Linux client?

ejpa at 2007-7-9 5:43:48 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...