I have very large queue about which contains messages about 1GB in size and above.
Can I configure Sun Java System Message Queue to swap incomming messages to disk if system has already NO FREE MEMORY. I use "-Xmx600m -Xms400m" and when I still sending messages to the queue the system will fail and it shows OutOfMemoryError!
Can somebody help me?
If you are running 3.5 with persistent messages then the in-memory copy of the message will be released when the system runs low on memory, but the system still maintains a reference to the message in memory so it occupies some space. If you have many, many small messages then the overhead of the message reference may be significant.
Non-persistent messages are always kept in memory.
JSMQ does not currently support a disk only backed destination (where messages only occupy disk space).
If you are running 3.0.1 imqbrokerd also releases the in-memory copy of persistent messages when memory gets low, but the mechanism is not as robust since it relies on imqbrokerd explicitly checking the JVM heap size.
I'm using version 3.5SP1 to specify the correct system version. I'm persisting messages with default settings, so that they may be persistent. I currently using file based backend storage.
Do you mean that:
1. I must use database storage for messages to persist messages?
in reply to:
JSMQ does not currently support a disk only backed destination (where messages only occupy disk space).
2. you say that if number of my stored messages gets over same large number so system fails because it keeps some references to persisted messages?
I have two questions:
-- My system after I was send to it about 200000 persistent messages to one queue is restarting very long time. In log file is shown message "[B1136]: Processing stored transactions" and system is starting many many minutes (20 minutes and it still not started). Do you have any advice about it?
-- May I set property imqConnectionFlowCount to say 1 or 10 messages? Because I sending persistent messages?
I'm currently trying to configure properties file but system is still behaving the same way. Here is my properties file :
imq.instanceconfig.version=300
imq.autocreate.queue=false
imq.autocreate.destination.maxNumMsgs=0
imq.autocreate.destination.maxTotalMsgBytes=0
imq.autocreate.destination.limitBehavior=REMOVE_OLDEST
imq.autocreate.destination.maxBytesPerMsg=100k
Here is my log file: Every thread accepting means sending 10000 ObjectMessages, to see the system speed.
# 1092163502968 Do not modify this line
[10/VIII/2004:20:45:02 CEST] [B1002]: An existing property file for imqbroker was not found, no stored properties will be loaded
[10/VIII/2004:20:45:03 CEST]
=============================================================================== =
Sun Java(tm) System Message Queue
Sun Microsystems, Inc.
Version: 3.5 SP1 (Build 48-G)
Compile: Thu 01/29/2004
Copyright 2004 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
This product includes code licensed from RSA Data Security.
=============================================================================== =
Java Runtime: 1.4.2_04 Sun Microsystems Inc. C:\Program Files\Sun\MessageQueue3\jre
[10/VIII/2004:20:45:03 CEST] License: Sun Java(tm) System Message Queue 3.5 SP1 PE License.
[10/VIII/2004:20:45:03 CEST]IMQ_HOME=C:\Program Files\Sun\MessageQueue3
[10/VIII/2004:20:45:03 CEST] IMQ_VARHOME=C:\Program Files\Sun\MessageQueue3\var
[10/VIII/2004:20:45:03 CEST] Windows XP 5.1 x86 localhost (1 cpu) JanVitsek
[10/VIII/2004:20:45:03 CEST] Java Heap Size: max=194432k, current=16256k
[10/VIII/2004:20:45:03 CEST] Arguments:
[10/VIII/2004:20:45:03 CEST] [B1004]: Starting the portmapper service using tcp [ 7676, 50, * ] with min threads 1 and max threads of 1
[10/VIII/2004:20:45:03 CEST] [B1060]: Loading persistent data...
[10/VIII/2004:20:45:03 CEST] Using built-in file-based persistent store: C:\Program Files\Sun\MessageQueue3\var\instances\imqbroker\
[10/VIII/2004:20:45:04 CEST] [B1136]: Processing stored transactions
[10/VIII/2004:20:45:04 CEST] [B1013]: Auto Creation of Queues is Enabled
[10/VIII/2004:20:45:04 CEST] [B1004]: Starting the jms service using tcp(host = *, port=0, mode=dedicated) with min threads 10 and max threads of 1000
[10/VIII/2004:20:45:04 CEST] [B1004]: Starting the admin service using tcp(host = *, port=0, mode=dedicated) with min threads 4 and max threads of 10
[10/VIII/2004:20:45:04 CEST] [B1039]: Broker "imqbroker@localhost:7676" ready.
[10/VIII/2004:20:46:08 CEST] [B1065]: Accepting: guest@127.0.0.1:1043->jms:1040. Count=1
[10/VIII/2004:20:46:08 CEST] [B1132]: Autocreating destination archivace [Queue]
[10/VIII/2004:20:46:33 CEST] [B1066]:Closing: guest@127.0.0.1:1043->jms:1040. Count=0
[10/VIII/2004:20:46:46 CEST] [B1065]: Accepting: guest@127.0.0.1:1045->jms:1040. Count=1
[10/VIII/2004:20:47:03 CEST] [B1066]:Closing: guest@127.0.0.1:1045->jms:1040. Count=0
[10/VIII/2004:20:47:14 CEST] [B1065]: Accepting: guest@127.0.0.1:1047->jms:1040. Count=1
[10/VIII/2004:20:47:30 CEST] [B1066]:Closing: guest@127.0.0.1:1047->jms:1040. Count=0
[10/VIII/2004:20:47:52 CEST] [B1065]: Accepting: guest@127.0.0.1:1049->jms:1040. Count=1
[10/VIII/2004:20:48:10 CEST] [B1066]:Closing: guest@127.0.0.1:1049->jms:1040. Count=0
[10/VIII/2004:20:48:24 CEST] [B1065]: Accepting: guest@127.0.0.1:1051->jms:1040. Count=1
[10/VIII/2004:20:48:35 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 116593K, 59% of total memory used
[10/VIII/2004:20:48:43 CEST] [B1066]:Closing: guest@127.0.0.1:1051->jms:1040. Count=0
[10/VIII/2004:20:56:58 CEST] [B1065]: Accepting: guest@127.0.0.1:1053->jms:1040. Count=1
[10/VIII/2004:20:57:14 CEST] [B1066]:Closing: guest@127.0.0.1:1053->jms:1040. Count=0
[10/VIII/2004:20:57:32 CEST] [B1065]: Accepting: guest@127.0.0.1:1055->jms:1040. Count=1
[10/VIII/2004:20:57:48 CEST] [B1066]:Closing: guest@127.0.0.1:1055->jms:1040. Count=0
[10/VIII/2004:20:58:08 CEST] [B1065]: Accepting: guest@127.0.0.1:1057->jms:1040. Count=1
[10/VIII/2004:20:58:26 CEST] [B1066]:Closing: guest@127.0.0.1:1057->jms:1040. Count=0
[10/VIII/2004:20:58:39 CEST] [B1065]: Accepting: guest@127.0.0.1:1059->jms:1040. Count=1
[10/VIII/2004:20:59:01 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 129524K, 66% of total memory used
[10/VIII/2004:20:59:01 CEST] [B1066]:Closing: guest@127.0.0.1:1059->jms:1040. Count=0
[10/VIII/2004:20:59:23 CEST] [B1065]: Accepting: guest@127.0.0.1:1061->jms:1040. Count=1
[10/VIII/2004:20:59:37 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 135379K, 69% of total memory used
[10/VIII/2004:20:59:44 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:20:59:44 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0021]: GREEN - allocated memory is 140700K, 72% of total memory used
[10/VIII/2004:20:59:46 CEST] [B1066]:Closing: guest@127.0.0.1:1061->jms:1040. Count=0
[10/VIII/2004:21:05:37 CEST] [B1065]: Accepting: guest@127.0.0.1:1063->jms:1040. Count=1
[10/VIII/2004:21:05:39 CEST] WARNING [B2011]: Storing of JMS message from IMQConn[AUTHENTICATED,guest@127.0.0.1:1063,jms:1040] failed:
com.sun.messaging.jmq.jmsserver.util.BrokerException: [B4120]: Can not store message 6-127.0.0.1(a6:f4:b:1c:d1:a0)-1063-1092164737671 on destination archivace [Queue] because capacity of 100000 would be exceeded.
[10/VIII/2004:21:18:36 CEST] [B1065]: Accepting: admin@127.0.0.1:1065->admin:1041. Count=2
[10/VIII/2004:21:19:23 CEST] [B1066]:Closing: guest@127.0.0.1:1063->jms:1040. Count=1
[10/VIII/2004:21:19:25 CEST] [B1065]: Accepting: guest@127.0.0.1:1067->jms:1040. Count=2
[10/VIII/2004:21:22:17 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 95760K, 49% of total memory used
[10/VIII/2004:21:22:41 CEST] [B1066]:Closing: guest@127.0.0.1:1067->jms:1040. Count=1
[10/VIII/2004:21:23:18 CEST] [B1065]: Accepting: guest@127.0.0.1:1069->jms:1040. Count=2
[10/VIII/2004:21:23:18 CEST] [B1132]: Autocreating destination mq.metrics.destination_list [Topic]
[10/VIII/2004:21:23:18 CEST] WARNING [B2007]: Creation of destination mq.metrics.destination_list failed:
com.sun.messaging.jmq.jmsserver.util.BrokerException: [B4180]: Support for feature [B0048]: Monitoring Destinations is unavailable on destination mq.metrics.destination_list [Topic], please upgrade to the Enterprise Edition to enable this feature
[10/VIII/2004:21:23:19 CEST] [B1066]:Closing: guest@127.0.0.1:1069->jms:1040. Count=1
[10/VIII/2004:21:24:52 CEST] [B1065]: Accepting: guest@127.0.0.1:1071->jms:1040. Count=2
[10/VIII/2004:21:25:13 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 128040K, 65% of total memory used
[10/VIII/2004:21:25:19 CEST] [B1066]:Closing: guest@127.0.0.1:1071->jms:1040. Count=1
[10/VIII/2004:21:26:13 CEST] [B1065]: Accepting: guest@127.0.0.1:1073->jms:1040. Count=2
[10/VIII/2004:21:26:16 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:21:26:16 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0021]: GREEN - allocated memory is 136518K, 70% of total memory used
[10/VIII/2004:21:26:33 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 123849K, 63% of total memory used
[10/VIII/2004:21:26:45 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:21:26:45 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0021]: GREEN - allocated memory is 137529K, 70% of total memory used
[10/VIII/2004:21:26:49 CEST] [B1066]:Closing: guest@127.0.0.1:1073->jms:1040. Count=1
[10/VIII/2004:21:30:21 CEST] [B1065]: Accepting: guest@127.0.0.1:1075->jms:1040. Count=2
[10/VIII/2004:21:30:30 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0023]: ORANGE - allocated memory is 124839K, 64% of total memory used
[10/VIII/2004:21:30:34 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 124839K, 64% of total memory used
[10/VIII/2004:21:30:45 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:21:30:45 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0021]: GREEN - allocated memory is 138535K, 71% of total memory used
[10/VIII/2004:21:30:52 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 148807K, 76% of total memory used
[10/VIII/2004:21:31:04 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 152230K, 78% of total memory used
[10/VIII/2004:21:31:17 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 153114K, 78% of total memory used
[10/VIII/2004:21:31:25 CEST] [B1066]:Closing: guest@127.0.0.1:1075->jms:1040. Count=1
[10/VIII/2004:21:34:06 CEST] [B1065]: Accepting: guest@127.0.0.1:1077->jms:1040. Count=2
[10/VIII/2004:21:34:19 CEST] [B1088]: Entering Memory State [B0021]: GREEN from previous state [B0022]: YELLOW - allocated memory is 131047K, 67% of total memory used
[10/VIII/2004:21:34:26 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:21:34:26 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0021]: GREEN - allocated memory is 137901K, 70% of total memory used
[10/VIII/2004:21:34:47 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 151563K, 77% of total memory used
[10/VIII/2004:21:34:59 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 154979K, 79% of total memory used
[10/VIII/2004:21:35:12 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 153312K, 78% of total memory used
[10/VIII/2004:21:35:24 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 149201K, 76% of total memory used
[10/VIII/2004:21:35:33 CEST] [B1066]:Closing: guest@127.0.0.1:1077->jms:1040. Count=1
[10/VIII/2004:21:37:30 CEST] [B1065]: Accepting: guest@127.0.0.1:1079->jms:1040. Count=2
[10/VIII/2004:21:38:27 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 154146K, 79% of total memory used
[10/VIII/2004:21:39:05 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 154874K, 79% of total memory used
[10/VIII/2004:21:39:41 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 152984K, 78% of total memory used
[10/VIII/2004:21:39:41 CEST] [B1066]:Closing: guest@127.0.0.1:1079->jms:1040. Count=1
[10/VIII/2004:21:41:56 CEST] [B1066]:Closing: admin@127.0.0.1:1065->admin:1041. Count=0
[10/VIII/2004:22:57:17 CEST] [B1065]: Accepting: guest@127.0.0.1:1081->jms:1040. Count=1
[10/VIII/2004:22:57:47 CEST] [B1088]: Entering Memory State [B0022]: YELLOW from previous state [B0023]: ORANGE - allocated memory is 153223K, 78% of total memory used
[10/VIII/2004:22:58:01 CEST] [B1089]: In low memory condition, Broker is attempting to free up resources
[10/VIII/2004:22:58:01 CEST] [B1088]: Entering Memory State [B0023]: ORANGE from previous state [B0022]: YELLOW - allocated memory is 156624K, 80% of total memory used
[10/VIII/2004:22:58:26 CEST] [B1088]: Entering Memory State [B0023]: ORANGE from previous state [B0024]: RED - allocated memory is 170012K, 87% of total memory used
[10/VIII/2004:22:58:49 CEST] [B1065]: Accepting: admin@127.0.0.1:1083->admin:1041. Count=2
[10/VIII/2004:22:58:57 CEST] [B1066]:Closing: guest@127.0.0.1:1081->jms:1040. Count=1
[10/VIII/2004:22:59:36 CEST] [B1125]: Pausing Destination archivace
[10/VIII/2004:22:59:53 CEST] [B1129]: Resuming Destination archivace
[10/VIII/2004:23:00:06 CEST] [B1065]: Accepting: guest@127.0.0.1:1085->jms:1040. Count=2
[10/VIII/2004:23:00:28 CEST] [B1088]: Entering Memory State [B0023]: ORANGE from previous state [B0024]: RED - allocated memory is 174344K, 89% of total memory used
[10/VIII/2004:23:01:07 CEST] [B1066]:Closing: admin@127.0.0.1:1083->admin:1041. Count=1
[10/VIII/2004:23:01:07 CEST] [B1088]: Entering Memory State [B0023]: ORANGE from previous state [B0024]: RED - allocated memory is 176741K, 90% of total memory used
[10/VIII/2004:23:01:40 CEST] [B1065]: Accepting: admin@127.0.0.1:1087->admin:1041. Count=2
[10/VIII/2004:23:01:42 CEST] [B1066]:Closing: guest@127.0.0.1:1085->jms:1040. Count=1
[10/VIII/2004:23:02:11 CEST] [B1066]:Closing: admin@127.0.0.1:1087->admin:1041. Count=0
[10/VIII/2004:23:15:14 CEST] [B1047]: Shutting down broker...
[10/VIII/2004:23:15:14 CEST] [B1007]: Stopping Service admin with protocol tcp(host = *, port=0, mode=dedicated)
[10/VIII/2004:23:15:14 CEST] [B1007]: Stopping Service jms with protocol tcp(host = *, port=0, mode=dedicated)
[10/VIII/2004:23:15:14 CEST] [B1077]: Broadcast good-bye to all connections ...
[10/VIII/2004:23:15:14 CEST] [B1078]: Flushing good-bye messages ...
[10/VIII/2004:23:15:29 CEST] [B1063]: Done
[10/VIII/2004:23:15:29 CEST] [B1048]: Shutdown of broker complete.
[10/VIII/2004:23:15:31 CEST]
=============================================================================== =
Sun Java(tm) System Message Queue
Sun Microsystems, Inc.
Version: 3.5 SP1 (Build 48-G)
Compile: Thu 01/29/2004
Copyright 2004 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
This product includes code licensed from RSA Data Security.
=============================================================================== =
Java Runtime: 1.4.2_04 Sun Microsystems Inc. C:\Program Files\Sun\MessageQueue3\jre
[10/VIII/2004:23:15:31 CEST] License: Sun Java(tm) System Message Queue 3.5 SP1 PE License.
[10/VIII/2004:23:15:31 CEST]IMQ_HOME=C:\Program Files\Sun\MessageQueue3
[10/VIII/2004:23:15:31 CEST] IMQ_VARHOME=C:\Program Files\Sun\MessageQueue3\var
[10/VIII/2004:23:15:31 CEST] Windows XP 5.1 x86 localhost (1 cpu) JanVitsek
[10/VIII/2004:23:15:31 CEST] Java Heap Size: max=194432k, current=16256k
[10/VIII/2004:23:15:31 CEST] Arguments:
[10/VIII/2004:23:15:31 CEST] [B1004]: Starting the portmapper service using tcp [ 7676, 50, * ] with min threads 1 and max threads of 1
[10/VIII/2004:23:15:31 CEST] [B1060]: Loading persistent data...
[10/VIII/2004:23:15:31 CEST] Using built-in file-based persistent store: C:\Program Files\Sun\MessageQueue3\var\instances\imqbroker\
[10/VIII/2004:23:15:32 CEST] [B1136]: Processing stored transactions
> Do you mean that:
> 1. I must use database storage for messages to persist messages?
No.
>2. you say that if number of my stored messages gets over same large
>number so system fails because it keeps some references to
>persisted messages?
Yes. Even though the system releases the in-memory copy of a message
when memory gets tight it still keeps references (with metadata) to
those messages in memory. When you have 200,000 messages this overhead
adds up and you can run out of memory.
According to the log file you are not increasing the Java heap size:
[10/VIII/2004:20:45:03 CEST] Java Heap Size: max=194432k, current=16256k
I'm not sure how you are starting the broker, but you can try
imqbrokerd -vmargs "-Xmx600m"
When imqbrokerd starts it checks to see if there are any open transactions.
If there are it must scan the persistent store to resolve those open
transactions. This can take a long time with 200,000 messages. If things
where shutdown cleanly then it shouldn't need to do this.
As to the tuning question, it all depends on what you want to do.
Do you want to limit the queue size? This is generally a good idea.
And what do you want to happenif the queue fills up?
Instead of using autocreated queues I would create them administratively.
For example if you want to set the queue size to 10,000 messages and
have the queue slow down the producer to match the rate at which
messages are being consumed you would create the queue on imqbrokerd with
a command like:
imqcmd create dst -n MyQueue -t q -o "maxNumMsgs=10000" -o "limitBehavior=FLOW_CONTROL"
Other behaviors are possible. For more information see the 3.5SP1
Administration Guide, Creating Destinations:
http://docs.sun.com/source/817-6024/brkrmgmt.html#wp36247
There should be no reason for you to change imqConnectionFlowCount.
Joe
http://wwws.sun.com/software/products/message_queue/
Yes you are right. :) On this test I started broker with default memory setting. On this test I'm sending only 10000 messages in one run on my test program. With high memory settings I was sending 30000 object messages with Hashtables.
I think that may be good for people which uses queues only for cases where they doesn't need message expirations to store this messages directly to the database and if memory exceeds some predefined top than it may delete this messages from memory. So this product will have STABILITY and RELIABILITY interchanging it FOR SPEED. For many application IT MAY BE VERY USEFULL.
I have one last question:
-- How do I stop (shutdown) the broker correctly to decrease the starting time consuming the processing stored transactions?
Many Thanks
Yes, we understand that a destination style or mode where messages only consume disk space would be useful to some customers.
As to your shutdown question. If you shutdown the broker with
imqcmd shutdown bkr
then it should shutdown cleanly and start up quickly. But the first time you send a message to the (very large) destination or attach a consumer the destination will load and this will take a while.
