Load balanced router hangs

Hello,

did anybody else also experience the following problem?

Problem:

The router of a load balanced service object sometimes and all of a sudden stops forwarding incoming requests to its instances, although all instances are available and free.

This finally causes the whole application to hang, since no further call to a method of that service object is being processed.

Observations:

The dump of the load balanced router during this situation indicates that none of the instances is being used (see below). Besides, the instrument MembersInUse has a value of 0.

CPU load of the nodemgr and the router process increase significantly.

There is no DB deadlock.

We have had this problem on several different load balanced service objects (at different times; with dialog durations message or transaction).

It does not depend on overall system load or on available memory resources - the problem can occur at any time.

We can solve this lock situation by restarting the router.

UDS 5.0.3, HP-UX 11

Thanks in advance for any help or hints on what may be going on.

Martin.

Thu Apr 27 09:55:14 : LoadBalance-Router

Thu Apr 27 09:55:14 : State ADDING_MEMBERS Members: 5 MessageList: 0

Thu Apr 27 09:55:14 : Versions: part 14 object 0

Thu Apr 27 09:55:14 : -Member-Map-

Thu Apr 27 09:55:14 :Id: 0x1 Max: 1 Cur: 0 XactId: [, 0]

Thu Apr 27 09:55:14 :Object: qqlo_Object

Thu Apr 27 09:55:14 :Id: 0x2 Max: 1 Cur: 0 XactId: [, 0]

Thu Apr 27 09:55:14 :Object: qqlo_Object

Thu Apr 27 09:55:14 :Id: 0x3 Max: 1 Cur: 0 XactId: [, 0]

Thu Apr 27 09:55:14 :Object: qqlo_Object

Thu Apr 27 09:55:14 :Id: 0x4 Max: 1 Cur: 0 XactId: [, 0]

Thu Apr 27 09:55:14 :Object: qqlo_Object

Thu Apr 27 09:55:14 :Id: 0x5 Max: 1 Cur: 0 XactId: [, 0]

Thu Apr 27 09:55:14 :Object: qqlo_Object

[1943 byte] By [mrrmuc] at [2007-11-26 6:48:08]
# 1

We used to have a similar problem on Solaris a while ago.

We found out that our company's security scans of our servers was giving our partitions problems. We suspended the security scans and I don't think we've seen the problem since.

We initially noticed the problem with 5.1.x partitions because there were some useful messages in the log files. However, later I found out that it was also affecting our 5.0.24 partitions but without the errors in the log files.

If you think your servers are the subject of security scans (or even hackers), I can provide some more information.

MarkCarruthers at 2007-7-6 15:11:42 > top of Java-index,Application & Integration Servers,Integration Servers...
# 2

Hi Mark,

thank you for your response. I talked to our network group, they told me, that there should be no scans going on. Do you have an idea how we could verify this from the application side, e.g. are there any trace flags we could enable to get some feedback?

Anyway, we did further investigations and found the following:

1) The nameservice contains references to old, non-existing partitions, even to partitions that lived before the last reboot of the system, which are not marked as "[Needs Rebind] [Rebinding]" (showpart in nameservice). By the way, the instrument DeleteOnCommFailure is set to TRUE in our environment.

2) The directory $FORTE_ROOT/tmp containing the unix domain socket files does not get cleaned up. It contains even very old entries.

3) Putting 1) and 2) together, we found out that during the hangs of our system, the load balanced router tries to send a message to one of the not-more-existing instances of the Service Object.

Example:

The following dumps were done while our system was hanging.

The router partition in question is ...:0x3ce25:0x1.

The non-existing partition of the Service Object instance is ...:0x3b913:0x2.

The non-existing process has the PID 16736 (there was such a process running a Service Object instance prior to rebooting the system).

The socket file '/forte/tmp/FORTE_A00004160' still exists.

An extract of 'showpart' on the nameservice agent:

xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913 realenv

Forte Runtime Forte_DCE_ExecutorExecId: 16736 (ftserver)

Locations:

Location Descriptor for protocol: Forte!UnixDomain

Path/forte/tmp/FORTE_A00004160

HostNameomega

Location Descriptor for protocol: Forte!TCP/IP

HostNameomega

PortNumber59128

DotAddressxxx.xxx.xxx.xxx

xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 realenv

Forte Runtime DBSharedDataEntities_cl0_Part2 ExecId: 16736 (ftserver)

Locations:

Location Descriptor for protocol: Forte!UnixDomain

Path/forte/tmp/FORTE_A00004160

HostNameomega

Location Descriptor for protocol: Forte!TCP/IP

HostNameomega

PortNumber59128

DotAddressxxx.xxx.xxx.xxx

A TaskMgr dump during the hang:

(notice the messages having as destination the non-existing Service Object instance partition ...:0x3b913:0x2).

Thu Apr 27 09:55:14 : 165 DOM Id8 PART_CREATEPARTid 0x3b913

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 166 INC id8 count3 proxymgr.ResolveOutGoingProxy

Thu Apr 27 09:55:14 : 167 DOM Id8 PART_STARTPARTid 0x3b913

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 168 DOM Id8 PART_CHECKSTARTid 0x3b913

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 169 DOM Id8 LOCSET_CREATELOCSET id 0x3b913 partid 0x0

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913Version: 0

Thu Apr 27 09:55:14 :PartitionsUsing: 1 FailedNumber: 13106

Thu Apr 27 09:55:14 : 170 INC id8 count3 dom.exchangemethod.1

Thu Apr 27 09:55:14 : 171 DOM Id8 MSG_ALLOCATEid 0x16edd local 0x15

Thu Apr 27 09:55:14 : 172 INC id8 count5 proxymgr.ResolveOutGoingProxy

Thu Apr 27 09:55:14 : 173 DEC id8 count3 proxymgr.ResolveOutGoingProxy

Thu Apr 27 09:55:14 : 174 DOM Id8 MSG_SENTid 0x16edd to partid 0x3cd65

Thu Apr 27 09:55:14 : 175 DOM Id8 MSG_COMPLETEDOMid 0x16edd

Thu Apr 27 09:55:14 : 176 DOM Id8 MSG_COMPLETECMid 0x16edd

Thu Apr 27 09:55:14 : 177 DOM Id8 MSG_POSTSENDid 0x16edd

Thu Apr 27 09:55:14 : 178 DEC id8 count2 dom.exchangemethod.2

Thu Apr 27 09:55:14 : 179 DOM Id8 MSG_SUSPEND id 0x16edd

Thu Apr 27 09:55:14 : 180 DOM Id18 MSG_ALLOCATEid 0x16ede local 0xa

Thu Apr 27 09:55:14 : 181 DOM Id18 MSG_BEGINDESERid 0x16ede

Thu Apr 27 09:55:14 : 182 DOM Id18 MSG_FOUND_REPLYid 0x16ede

Thu Apr 27 09:55:14 : 183 INC id18 count1 domsg.process.1

Thu Apr 27 09:55:14 : 184 DOM Id18 MSG_RECEIVE id 0x16ede reply msg 0x16edd

Thu Apr 27 09:55:14 : 185 DOM Id18 MSG_ACK id 0x16edd

Thu Apr 27 09:55:14 : 186 DOM Id18 MSG_RESUME id 0x0 resuming task 0x40166638

Thu Apr 27 09:55:14 : 187 DEC id18 count0 domsg.process.8

Thu Apr 27 09:55:14 : 188 DOM Id8 MSG_FREEid 0x16edd local 0x15

Thu Apr 27 09:55:14 : 189 DOM Id8 MSG_FREEid 0x16ede local 0xa

Thu Apr 27 09:55:14 : 190 DOM Id8 LOC_ADDLOCATIONid 0x938 partid 0x3b913

Thu Apr 27 09:55:14 :0x938 State: DESTROY Adv OwningSubPart: 0

Thu Apr 27 09:55:14 : PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913

Thu Apr 27 09:55:14 : CM DestPartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Unix Domain Socket Location - Host: omega Path: /forte/tmp/FORTE_A00004160

Attributes:

Path /forte/tmp/FORTE_A00004160

HostName omega

Thu Apr 27 09:55:14 : CountersSend: 1 Receive: 0

Thu Apr 27 09:55:14 : 191 INC id8 count3 dom.exchange.1

Thu Apr 27 09:55:14 : 192 DOM Id8 MSG_ALLOCATEid 0x16edf local 0xa

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 :

Thu Apr 27 09:55:14 : 193 DOM Id8 MSG_SENTid 0x16edf to partid 0x3b913

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 : 194 DOM Id13 LOC_STOPLOCATIONid 0x938 partid 0x3b913

Thu Apr 27 09:55:14 :0x938 State: DESTROY Adv OwningSubPart: 0

Thu Apr 27 09:55:14 : PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913

Thu Apr 27 09:55:14 : CM DestPartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Unix Domain Socket Location - Host: omega Path: /forte/tmp/FORTE_A00004160

Attributes:

Path /forte/tmp/FORTE_A00004160

HostName omega

Thu Apr 27 09:55:14 : CountersSend: 1 Receive: 0

Thu Apr 27 09:55:14 : 195 DOM Id13 LOC_DESTLOCATIONid 0x938 partid 0x3b913

Thu Apr 27 09:55:14 : Thu Apr 27 09:55:14 :0x938 State: DESTROY Adv OwningSubPart: 0

Thu Apr 27 09:55:14 : PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913

Thu Apr 27 09:55:14 : CM DestPartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Unix Domain Socket Location - Host: omega Path: /forte/tmp/FORTE_A00004160

Attributes:

Path /forte/tmp/FORTE_A00004160

HostName omega

Thu Apr 27 09:55:14 : CountersSend: 1 Receive: 0

Thu Apr 27 09:55:14 : 196 DOM Id13 LOCSET_DESTLOCSETid 0x3b913 partid 0x0

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913Version: 0

Thu Apr 27 09:55:14 :PartitionsUsing: 1 FailedNumber: 13106

Thu Apr 27 09:55:14 :

Thu Apr 27 09:55:14 : 197 INC id13 count2 partmgr.stoplocation.1

Thu Apr 27 09:55:14 : 198 DOM Id13 PART_DESTREMINTid 0x3b913 sub 0x2

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 199 DOM Id13 PART_FAULTNOPROXYid 0x3b913

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 200 DOM Id13 PART_PARTLOSTid 0x0 partid 0x3b913

Thu Apr 27 09:55:14 : 201 DOM Id13 PART_FINCONNLOSTid 0x3b913

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 202 INC id13 count2 dom.exchange.1

Thu Apr 27 09:55:14 : 203 DOM Id13 MSG_ALLOCATEid 0x16ee0 local 0x15

Thu Apr 27 09:55:14 : 204 DOM Id13 MSG_SENTid 0x16ee0 to partid 0x3b913

Thu Apr 27 09:55:14 : 205 DOM Id13 PART_SENDFAILid 0x0 error 0x3b913

Thu Apr 27 09:55:14 : 206 DOM Id13 MSG_FAILEDDOMid 0x16ee0 Message::Send.1

Thu Apr 27 09:55:14 : 207 DOM Id13 MSG_FREEid 0x16ee0 local 0x15

Thu Apr 27 09:55:14 : 208 DEC id13 count1 Dom.exchange.2

Thu Apr 27 09:55:14 : 209 DOM Id13 PART_FINREMOVEid 0x3b913 sub 0x2

Thu Apr 27 09:55:14 :xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2 DESTROY - OUTBOUND

Thu Apr 27 09:55:14 : 210 DEC id13 count0 partmgr.stoplocation.3

Thu Apr 27 09:55:14 : 211 DOM Id8 MSG_COMPLETEDOMid 0x16edf

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 : 212 DOM Id8 MSG_COMPLETECMid 0x16edf

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 : 213 DOM Id8 MSG_POSTSENDid 0x16edf

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 :

Thu Apr 27 09:55:14 : 214 DOM Id8 MSG_SUSPEND id 0x16edf

Thu Apr 27 09:55:14 :0x16edf-0xa [NETWORK] (EXCHANGE MESSAGE NEEDS_REPLY)

Thu Apr 27 09:55:14 : ReplyMsg:0x0-0x0 RelatedMsg: 0x0-0x0 OrigMsgSeq: 0x16edf

Thu Apr 27 09:55:14 : Status:OUTBOUND TASK_BLOCKED

Thu Apr 27 09:55:14 : Previous:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Originating: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : Destination: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : Next:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3b913:0x2

Thu Apr 27 09:55:14 : ToUse:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 : TaskId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

Thu Apr 27 09:55:14 : AssocTask:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1, 10

0x401af4f0

Thu Apr 27 09:55:14 : Location:0x0 SavedLocation: 0x00000000 LocaleId: 0x0

Thu Apr 27 09:55:14 : SpecialOpt: BRANCH_SYNC

Thu Apr 27 09:55:14 : Version:0 FailedNum: 0 Protocol: 0

Thu Apr 27 09:55:14 : InternalMessage PROXY_VALIDATE

PartId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:0x3ce25:0x1

Thu Apr 27 09:55:14 :Adv ProxyId: 0x27 Ref ProxyId: 0x0 Duration: 0

Scope/Class 0x0:0x0 Proxy Status: 0x0

Version - Obj: 0 Pxy: 0

Thu Apr 27 09:55:14 :ToPartId: 0x0

Thu Apr 27 09:55:14 :

Thu Apr 27 09:55:14 : 215 DEC id8 count2 dom.exchange.3

mrrmuc at 2007-7-6 15:11:42 > top of Java-index,Application & Integration Servers,Integration Servers...
# 3

I don't know about any trace flags that would help diagnose security scan/hacker related problems. As I said earlier, with Forte 5.1.x we got messages in the logs that indicated a problem. Here's an example:

Task 6: ERROR: qqcm_Hose::ReceiveComplete: Bad packet length 2048 after 16 bytes read of 2048 possible. Packet is corrupt or not from a Forte partition. Make sure no one is attempting erroneous connects to this location: Internet Location - Host: hostnamePort Number: xxxx Dot: xxx.xx.xxx.xx

However, we didn't get any useful error messages in the Forte 5.0.x logs.

It sounds like you may have a different problem.

Good luck.

MarkCarruthers at 2007-7-6 15:11:42 > top of Java-index,Application & Integration Servers,Integration Servers...