Message Delivery in a Clustered Environment
Hello,
I was wondering if someone could give me a quick explanation/lesson on the following scenario.
I have a two broker cluster (say, SERVERA and SERVERB where SERVERA is the master). Each server has a consumer connected to it, CONA and CONB respectively. I'm throwing evenly distributed producers at the system such that message consumption cannot keep up with production.
CONB is slow due to issues external to the JMS system. After 15 minutes, when I stop the producers, SERVERB has ~3,500 messages in the queue whereas SERVERA has ~450.CONA stops receiving messages when there is still 2750 messages in the queue on SERVERB (SERVERA has 0 left). CONB, which is slow, is left to deal with the remaining messages.Restarting the CONA seems to work and messages being flowing from SERVERB to CONA via SERVERA.
I've tried a consumerFlowLimit of 1000 and 100 but the same thing is happening.I also don't have localDeliveryPreferred set. My goal is to have both CONA and CONB working until all messages in the cluster have been processed. Is there a config setting I can tweak to fix this?
Thanks in advance,
Aaron
[1166 byte] By [
aeddydev] at [2007-11-25 17:04:54]
# 1
This sounds like a bug. If consumerFlowLimit is set to 100 then no more than 100 messages should be "predelivered" to a consumer. Also the fact that the restarted consumer starts to get messages leads me to belive that this is not a simple issue of too many messages getting predelivered to the slow consumer.
That being said, you can still try to set the imqConsumerFlowLimit attribute on the ConnectionFactory of the slow consumer to something low (like 10) -- leaving the consumerFlowLimit destination attribute higher. This should enforce the bias to the fast consumer and maybe it will help.
I know you are using two brokers for a reason, but I'm curious to know if all producers and consumers are on the same broker do you still see this problem.
jfdp at 2007-7-3 0:18:30 >
# 2
Hi,
> I know you are using two brokers for a reason, but
> I'm curious to know if all producers and consumers
> are on the same broker do you still see this problem.
No. Messages are distributed as expected and both consumers are working right up until all messages have been consumed.
Thanks for your response.
Aaron
# 3
What is the MQ broker version and the MQ client version
(if different from broker) ?
Steps (when/order in starting brokers, producer, consumers)
to reproduce the problem ?
Can you provide code examples of the consumers and
producer that reproduces the problem ? This will
also answer the questions like,
Is the destination Queue or Topic ?
Are the messages produced persistent messages ?
Are the consumers using transacted session or what
is acknowledge mode ?
..
You can provide the information to imq-feedback@sun.com
amy
http://wwws.sun.com/software/products/message_queue/
# 4
Thanks for your response.
I'll answer your questions as best as I can remember but I can't invest any time in recreating the situation. Since this post, we have set localDeliveryPreferred because the amount of inter-broker communication seemed too much. The server with the slow consumer received 6110 messages according to metrics and the fast server received 4950 messages. Each were sent 4080 messages.
> What is the MQ broker version and the MQ client
> version
> (if different from broker) ?
They should have both been running 3.5 SP1. However, it is possible that the client was 3.5.
> Steps (when/order in starting brokers, producer,
> consumers)
> to reproduce the problem ?
Start cluster brokers.
Start consumers.
Start producers.
After the one consumer stopped receiving messages, I restarted it and messages began flowing again.
> Can you provide code examples of the consumers and
> producer that reproduces the problem ? This will
> also answer the questions like,
> Is the destination Queue or Topic ?
Queue.
> Are the messages produced persistent messages ?
Persistent.
> Are the consumers using transacted session or what
> is acknowledge mode ?
DUPS_OK_ACKNOWLEDGE
Aaron
