Secure LDAP with Multiple DPS's on Single Physical Server

I am having an issue connecting to the directory server over SSL via the directory proxy server. I have enabled SSL and tested successfully in some situations, however this situation is unique.

DPS 5.2 patch 4

Directory Server patch 4

I have applied the neccessary hotfixes from sun to resolve the SSL issues.

There are 3 physical servers. 2 of those servers each have 2 instances of directory proxy server running. The 3rd server has 2 separate Directory Server instances running (1 for enterprise authentication, 1 for Access Manager). Each physical proxy server has 1 instance running for each Directory server instance (1 enterprise LDAP, 1 AM LDAP). All 4 proxy instances can connect successfully to the Directory Masters over the unsecure ports (389 for enterprise LDAP, 55389 for AM LDAP). On the proxy servers, only the initial proxy instance can connect to the secure port successfully. For instance, on server 1 first the DPS for enterprise LDAP was installed then a DPS for AM LDAP was added. Only the enterprise proxy instance can connect successfully over SSL. On server 2 initially the AM LDAP instance was installed and then an enterprise LDAP instance was added. In this case only the AM LDAP instance can connect successfully.

For both instances of the proxy the appropriate certificates have been installed and verified. I can use the dps-instance-cert8.db for the working and non working DPS instances and successfully connect to the Directory Master using ldapsearch from the directory proxy server.

When I do ldapsearch I receive the following error on the second instances:

ldap_simple_bind: Can't contact LDAP server

SSL error -12271 (SSL peer cannot verify your certificate.)

Certificates:

bash-2.05$ /jes/ds52/shared/bin/certutil -L -d /jes/ds52/alias -P dps-instance1-

LDAP Development Pu,u,u

CMS SUN CERTIFICATE AUTH 2023 CT,,

bash-2.05$ /jes/ds52/shared/bin/certutil -L -d /jes/ds52/alias -P dps-instance2-

LDAP Development Pu,u,u

CMS SUN CERTIFICATE AUTH 2023 CT,,

Each use the same server cert, the host is ?.test.com? using the asterisk so the hostname shouldn抰 matter.

DPS INSTANCE 1 ?Success

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385609] TCP_NODELAY was set on socket 3

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [310200] Success with enabling socket 16 for blocking

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [323705] ( xxx.xx.xxx.xx+ 636) syncConnection success.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385609] TCP_NODELAY was set on socket 16

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [TRACE][520503] Connection established to condo101.cms.hhs.gov

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300771] Promoting socket 16 via socket 1.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300751] Socket 16, success with SSL_HANDSHAKE_AS_CLIENT

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385701] Success with sessionPromote to SSL for socket 16.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385704] Success with setting SSL_AuthCertificateHook callback

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385705] Success with setting SSL_BadCertHook callback

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385706] Success with setting SSL_HandshakeCallBack

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [302019] Success with SSL_SetPKCS11PinArg (socket 16)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385748] SSL_SetURL skipped on socket 16 (null url)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385803] Success with SSL_ResetHandshake as client (socket 16)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385739] Certificate possesses valid times on socket 16

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385745] For socket 16, pinArg does possess a value.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [302024] Success with CERT_VerifyCertNow (checking signature, usage: "certUsageSSLServer").

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385725] Certificate accepted on socket 16

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300754] Success with handshake on socket 16

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385746] SSL_ForceHandshake success on socket 16

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [TRACE][171210] [client(xxx.xxx.xxx.xxx,3)] [server( xxx.xx.xxx.xx+ 636, 16)] Connection via SSL session

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [STAT/CONN][171211] [client(xxx.xxx.xxx.xxx,3)] Accepting connection via network-group-1

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [310200] Success with enabling socket 3 for blocking

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300771] Promoting socket 3 via socket 0.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300750] Socket 3, success with SSL_HANDSHAKE_AS_SERVER

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385701] Success with sessionPromote to SSL for socket 3.

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385704] Success with setting SSL_AuthCertificateHook callback

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385705] Success with setting SSL_BadCertHook callback

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385706] Success with setting SSL_HandshakeCallBack

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300801] Success with setting SSL_REQUEST_CERTIFICATE (1)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300802] Success with setting SSL_REQUIRE_CERTIFICATE (0)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300405] Success with SSL configuration on socket 3

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385803] Success with SSL_ResetHandshake as server (socket 3)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300406] Success with SSL promotion on socket 3

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [TRACE][390307] [client(xxx.xxx.xxx.xxx,3)] [server( xxx.xx.xxx.xx+ 636, 16)] Success with OnSSLEstablished rule... continuing

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [300754] Success with handshake on socket 3

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385713] Read on socket 3. Received 42 byte(s)

May 10 2007 09:44:18 server123 SunONEDPS[ 24710]: [DETAIL_TRACE] [385716] ber_get_next (socket 3) returned complete PDU

DPS INSTANCE 2 ?FAILING

ldapsearch -h server123 -p 55636 -P /<serverroot>/alias/dps-server123-cert8.db -D "cn=directory manager" -s base -w adminjes -b dc=cms,dc=hhs,dc=gov objectclass=*

ldap_simple_bind: Can't contact LDAP server

SSL error -12271 (SSL peer cannot verify your certificate.)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][300901] Successful match of xxx.xxx.xxx.xxx+36383 against ALL

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][110999] In permit_connection_from_ip(), The counter for IP:xxx.xxx.xxx.xxx is now 2 and the limit is 0

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385609] TCP_NODELAY was set on socket 15

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [310200] Success with enabling socket 16 for blocking

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [323705] ( xxx.xx.xxx.xx+55636) syncConnection success.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385609] TCP_NODELAY was set on socket 16

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][520503] Connection established to condo101.cms.hhs.gov

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300771] Promoting socket 16 via socket 1.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300751] Socket 16, success with SSL_HANDSHAKE_AS_CLIENT

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385701] Success with sessionPromote to SSL for socket 16.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385704] Success with setting SSL_AuthCertificateHook callback

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385705] Success with setting SSL_BadCertHook callback

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385706] Success with setting SSL_HandshakeCallBack

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [302019] Success with SSL_SetPKCS11PinArg (socket 16)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385748] SSL_SetURL skipped on socket 16 (null url)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385803] Success with SSL_ResetHandshake as client (socket 16)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385739] Certificate possesses valid times on socket 16

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385745] For socket 16, pinArg does possess a value.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [302024] Success with CERT_VerifyCertNow (checking signature, usage: "certUsageSSLServer").

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385725] Certificate accepted on socket 16

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300754] Success with handshake on socket 16

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385746] SSL_ForceHandshake success on socket 16

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][171210] [client(xxx.xxx.xxx.xxx, 15)] [server( xxx.xx.xxx.xx+55636, 16)] Connection via SSL session

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [310200] Success with enabling socket 15 for blocking

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300771] Promoting socket 15 via socket 0.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300750] Socket 15, success with SSL_HANDSHAKE_AS_SERVER

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385701] Success with sessionPromote to SSL for socket 15.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385704] Success with setting SSL_AuthCertificateHook callback

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385705] Success with setting SSL_BadCertHook callback

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385706] Success with setting SSL_HandshakeCallBack

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300801] Success with setting SSL_REQUEST_CERTIFICATE (1)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300802] Success with setting SSL_REQUIRE_CERTIFICATE (1)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300405] Success with SSL configuration on socket 15

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385803] Success with SSL_ResetHandshake as server (socket 15)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [300406] Success with SSL promotion on socket 15

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][390307] [client(xxx.xxx.xxx.xxx, 15)] [server( xxx.xx.xxx.xx+55636, 16)] Success with OnSSLEstablished rule... continuing

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [NOTICE][385721] Read on socket 15 failed.

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [NOTICE][385721] SSL_ERROR_BASE + 3, NSPR error: -12285 (0xffffd003). Native errno is: 11

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385714] ber_get_next (socket 15) returned LBER_DEFAULT

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [385714] SSL_ERROR_BASE + 3, NSPR error: -12285 (0xffffd003). Native errno is: 11

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [EXCEPTION][301006] Unexpected error on socket 15. (Error: -12285).

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][190401] [server( xxx.xx.xxx.xx+55636, 16)] Input was not a BER encoding or connection closed: source( xxx.xxx.xxx.xxx, 15)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][190401] [server( xxx.xx.xxx.xx+55636, 16)] SSL_ERROR_BASE + 3, NSPR error: -12285 (0xffffd003). Native errno is: 11

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [171505] [server( xxx.xx.xxx.xx+55636, 16)] Entering recycle_inner_connection

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [301201] Closing connection to:xxx.xx.xxx.xx+55636 (socket 16)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [DETAIL_TRACE] [301201] Closing connection to: xxx.xxx.xxx.xxx (socket 15)

May 10 2007 10:05:17 server123 SunONEDPS[ 26275]: [TRACE][110998] In done_connection_from_ip(), The counter for IP:xxx.xxx.xxx.xxx is now 1 and the limit is 0

[13663 byte] By [nick50119a] at [2007-11-27 3:59:29]
# 1

I have resolved the issue, it was simply a configuration issue and was not at all related to multiple proxy instances.

The following entry was found in the master server log:

[10/May/2007:10:46:48 -0400] - ERROR<4190> - Authentication - conn=-1 op=-1 msgId=-1 - Internal error Internal search base="CN=*.abc.123.gov, OU=OFFICE OF INFORMATION SERVICES, O=BUSINESS NAME, L=BALTIMORE, ST=MARYLAND, C=US" scope=2 filter=objectclass=*, Result : 32 (No such object)

I then found that the "Require a client certificate" flag was set in the admin console, after unchecking this option everything worked as expected.

nick50119a at 2007-7-12 9:04:01 > top of Java-index,Web & Directory Servers,Directory Servers...
# 2

In these situations it's important to separate SSL level issues from LDAP issues. I've had good results using "openssl s_client" to investigate and debug SSL/TLS level issues. I point it at the LDAP server and it sets up a connection handling the SSL layers and giving lots of diagnostics about SSL and the X.509 certificates in use.

Try looking for things like whether the SubjectAltName and/or Issuer's CN contain the same hostname you are using to connect.

After giving you the diagnostics it will be waiting with your stdin/stdout connected to the application via SSL (which could make you think it's hanging). Since you're probably not going to be typing the LDAP protocol by hand you want to abort s_client at this point (or type something that Directory Server will reject as a LDAP protocol error).

You can get openssl from the usual places: blastwave.org & sunfreeware.com.

Hopefuly this will help,

-Scott-

Scott.R.Corzinea at 2007-7-12 9:04:01 > top of Java-index,Web & Directory Servers,Directory Servers...