Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

  • Isomorphic
    replied
    The growth of RMQ Queues without limit has been bothering me as well. I've posted a question about this to the RabbitMQ community - check it out here: https://groups.google.com/forum/#!to...rs/0zXgBh-nX28. Let's see where this goes. We can definitely fix this by cycling Connections but this is sub-optimal given my reading of the JMS spec. FYI the current logic does work properly with ActiveMQ - not that I'm suggesting you go back to it :)

    In terms of the amqp thing: I think the issue is we're relying on Java serialization because we send our messages as type ObjectMessage. What we'll do is provide a config flag that switches to the TextMessage type which is consumable by things other than Java - which is often the whole point of using a broker. We'll probably just encode the payload via JSON to the String datatype that TextMessage requires and doc that. What do you think?

    Leave a comment:


  • claudiobosticco
    replied
    Thanks, I'll wait for the documentation update to see if that's something different from my setup.

    About post #32, did you notice the same behaviour of the queues on the rabbitMQ broker?

    I mean the fact that if I don't add these attributes:
    Code:
    amqp="true"
    amqpExchangeName="jms.durable.topic"
    amqpRoutingKey="updateOrdineChannel"
    amqpQueueName="updateOrdineChannel"
    it creates a new queue with a single consumer for every subscribe operation and for every re-connection (after the TTL expires).

    But with those attributes I've got the error "Message not Text or Bytes".

    BTW I've got the same error with the realtime messaging if I define a isc_DebugTarget queue.
    Last edited by claudiobosticco; 9 Nov 2017, 01:48.

    Leave a comment:


  • Isomorphic
    replied
    The contents of that PDF have been subsumed into the reference docs here: https://www.smartclient.com/smartcli...oup..messaging. This will be updated in the next couple of days with some typical configuration options for ActiveMQ and RabbitMQ and Tomcat - basically the context.xml configuration and server.properties settings to use those <Resource> blocks, but you've already sorted this out on your own so while I don't think there will be new content for you there, those who are just starting out will find it helpful.

    Leave a comment:


  • claudiobosticco
    replied
    Originally posted by Isomorphic View Post
    Regarding the question [cut]
    Thanks for your comments and explanations.

    Now I'm trying to define my application channels on the broker, to avoid the creation of a topic for every 'subscribe' operation on the same channel (which is normal if I've understood correctly).

    So, using the rabbitMQ management application I've defined a queue updateOrdineChannel of type durable, with a binding to the jms.durable.topic exchange.

    I've changed the resource definition like this:
    Code:
    <Resource
                name="jms/updateOrdineChannel"
                type="javax.jms.Topic"
                factory="com.rabbitmq.jms.admin.RMQObjectFactory"
                destinationName="updateOrdineChannel"
    
                amqp="true"
                amqpExchangeName="jms.durable.topic"
                amqpRoutingKey="updateOrdineChannel"
                amqpQueueName="updateOrdineChannel"
        />
            <ResourceLink global="jms/updateOrdineChannel" name="jms/updateOrdineChannel"/>
    because without those amqp attributes it seems that the queue has always zero consumers, and those others topics mentioned above are still created.

    Instead, with those attributes, the queue has the expected number of consumers, but the messaging fails with this error:
    Code:
    2017-11-08 16:53:27,382 DEBUG [claudio.bosticco@juventus.com 798] MessagingConnectionHandler Delivering data: [
        {
            TIPO_EVENTO:"SERIE A TIM",
    .....
            HA_RICHIESTO_POSTI_CENTRALI:false
        }
    ] to: com.isomorphic.messaging.JMSMessageDispatcher, channels: [
        "jms/updateOrdineChannel"
    ] 
    2017-11-08 16:53:27,402 DEBUG [claudio.bosticco@juventus.com 798] RPCDMI Invocation threw exception 
    com.rabbitmq.jms.util.RMQJMSException: Unsupported message type for AMQP destination
        at com.rabbitmq.jms.client.RMQMessageProducer.sendAMQPMessage(RMQMessageProducer.java:303)
        at com.rabbitmq.jms.client.RMQMessageProducer.internalSend(RMQMessageProducer.java:274)
        at com.rabbitmq.jms.client.RMQMessageProducer.access$200(RMQMessageProducer.java:32)
        at com.rabbitmq.jms.client.RMQMessageProducer$PreferMessageProducerPropertySendingStategy.send(RMQMessageProducer.java:429)
        at com.rabbitmq.jms.client.RMQMessageProducer.send(RMQMessageProducer.java:206)
        at com.isomorphic.messaging.JMSMessageDispatcher.deliver(JMSMessageDispatcher.java:194)
        at com.isomorphic.messaging.MessagingConnectionHandler.send(MessagingConnectionHandler.java:85)
        at com.isomorphic.rpc.BuiltinRPC.messagingSend(BuiltinRPC.java:1297)
        .....
    Caused by: java.lang.UnsupportedOperationException: MessageProducer.send to AMQP resource: Message not Text or Bytes
        ... 94 more
    2017-11-08 16:53:27,407 WARN  [claudio.bosticco@juventus.com 798] RequestContext rpcRequest.execute() failed:  
    com.rabbitmq.jms.util.RMQJMSException: Unsupported message type for AMQP destination
        at com.rabbitmq.jms.client.RMQMessageProducer.sendAMQPMessage(RMQMessageProducer.java:303)
        at com.rabbitmq.jms.client.RMQMessageProducer.internalSend(RMQMessageProducer.java:274)
        at com.rabbitmq.jms.client.RMQMessageProducer.access$200(RMQMessageProducer.java:32)
        at com.rabbitmq.jms.client.RMQMessageProducer$PreferMessageProducerPropertySendingStategy.send(RMQMessageProducer.java:429)
        at com.rabbitmq.jms.client.RMQMessageProducer.send(RMQMessageProducer.java:206)
        at com.isomorphic.messaging.JMSMessageDispatcher.deliver(JMSMessageDispatcher.java:194)
        at com.isomorphic.messaging.MessagingConnectionHandler.send(MessagingConnectionHandler.java:85)
    I've tried to add the
    Code:
    x-opt-jms-msg-type: TextMessage
    argument to the queue, but the error remains the same.

    Does it has something to do with the realtime messaging module? Or am I missing/misusing something? Any hints?

    Leave a comment:


  • claudiobosticco
    replied
    as a side note, is it available a recent version of Messaging_QuickRef.pdf ? Mine is pretty old, version 6.5.

    Leave a comment:


  • Isomorphic
    replied
    Regarding the question about duplicates: what you're seeing is normal and does not indicate duplicates. Each JVM (Tomcat instance in this case) gets on JMS connection to the broker. For a broadcast on any given channel, all connections will get a copy of the message (so both of your Tomcat instances log the fact that they received the message at DEBUG threshold - this logging wasn't present previously). But as you can see only the first tomcat instance actually delivered the message to the client (browser) - this is indicated by the other logs.

    The client-side logging that notes a duplicate is still in place, so you'd see it in the client side logs if that occurred. Note however that some duplicates are normal because due to firewalls and such, we have a connectionTTL setting (defaults to 2 minutes) which forces the browser to re-establish a connection. During re-establishment, the original connection is maintained to ensure continuity of message delivery and within this window a message may be delivered both on the original (closing) connection and on the newly established connection. This, in addition to simple defensive programming, is why we have a dup check client-side.

    Regarding the question on durable topics. Yes, this is normal. We generate a unique ID (that long Routing Key) for both the DeveloperConsole browser page and the page being debugged. This is turned into a durable topic of the same name. Because these IDs are pre-generated, we can't use a pre-configured durable topic (without a bunch of reworking of how the Remote Debugging code works). We can't use a temporary topic either because by spec, those will only deliver to the JMS connection on which they were created. This last bit is why you saw the issue with Remote Debugging only working properly when both browser instance were connected to the same Tomcat instance. Because we used to use temporary topics, but by definition two separate JVMs must have two different JMS connections, so the routing would fail for non-preconfigured durable topics/queues.

    My understanding is that JMS topics/queues that are dynamically created via the JMS session.createTopic() API (which is what we do unless a preconfigured Topic is available at the specified channelName) should be automatically deleted when the last consumer and producer disconnect. However, this won't happen if no messages are ever sent to the topic (which will happen if you have remote debugging enabled but never select that browser to be debugged). I believe they should go away when you restart the broker. And keep in mind that this is a developer tool - so this wouldn't be an issue in a production setting, because presumably there you use pre-configured durable topics/queues for communication and these ad-hoc channels wouldn't exist.

    Leave a comment:


  • claudiobosticco
    replied
    Another doubt.
    Note that I'm no JMS nor RabbitMQ expert.
    In the Exchanges tab of the rabbitMQ management console, if I click on jms.durable.topic, I find what I think are the jmsMessages sent.
    Please see the first 5 in the attachment.
    Click image for larger version

Name:	2017-11-07 17.58.29.jpg
Views:	170
Size:	81.4 KB
ID:	250216

    These topics are durable (I can't even Unbind them), is it normal? After some testing, that list has 1300+ rows.
    Attached Files

    Leave a comment:


  • claudiobosticco
    replied
    SmartClient Version: v11.1p_2017-11-05/Enterprise Deployment (built 2017-11-05)

    Hello, I can confirm that the remote debug is working, thank you very much.

    Regarding the "normal" messaging, I noticed that when I call
    Code:
    isc.Messaging.send("jms/updateOrdineChannel", [{foo: "bar"}])
    in the tomcat logs (of the node serving this request) I see:
    Code:
    2017-11-07 17:25:56,582 DEBUG [claudio.bosticco@juventus.com 798] RPCManager Processing 1 requests.
    2017-11-07 17:25:56,582 DEBUG [claudio.bosticco@juventus.com 798] RPCManager Request #1 (RPCRequest) data: {
        appID:"isc_builtin",
        className:"builtin",
        methodName:"messagingSend",
        arguments:[
            {
                type:"send",
                sendToChannels:[
                    "jms/updateOrdineChannel"
                ],
                subscribedChannels:{
                    "91A94D4E-2FDE-41A1-96A1-6AC77B79431F":{
                    },
                    isc_DebugTarget:{
                    },
                    "jms/removeOrdineChannel":{
                        subscriptionCallback:null
                    },
                    "jms/updateOrdineChannel":{
                        subscriptionCallback:null
                    },
                    "jms/addOrdineChannel":{
                        subscriptionCallback:null
                    }
                },
                data:[
                    {
                        foo:"bar"
                    }
                ]
            }
        ],
        is_ISC_RPC_DMI:true
    }
    2017-11-07 17:25:56,583 INFO  [claudio.bosticco@juventus.com 798] IDACall Performing 1 operation(s)
    2017-11-07 17:25:56,584 DEBUG [claudio.bosticco@juventus.com 798] MessagingConnectionHandler Delivering data: [
        {
            foo:"bar"
        }
    ] to: com.isomorphic.messaging.JMSMessageDispatcher, channels: [
        "jms/updateOrdineChannel"
    ]
    2017-11-07 17:25:56,587 DEBUG [claudio.bosticco@juventus.com 798] RPCDMI rpc returned RPCResponse
    2017-11-07 17:25:56,587 DEBUG [claudio.bosticco@juventus.com 798] JMSMessageDispatcher Decoded jmsMessage for channel: jms/updateOrdineChannel data[
        {
            foo:"bar"
        }
    ]
    2017-11-07 17:25:56,587 DEBUG [claudio.bosticco@juventus.com 798] RPCManager Content type for RPC transaction: text/plain; charset=UTF-8
    2017-11-07 17:25:56,588 DEBUG [claudio.bosticco@juventus.com ] MessagingConnectionHandler Sending message: ID:debbd9ca-f6ae-4cee-bf6e-be08b04bf23b:[
        {
            foo:"bar"
        }
    ]
    while in the logs of the other tomcat I see:
    Code:
    2017-11-07 17:25:56,585 DEBUG [ ] JMSMessageDispatcher Decoded jmsMessage for channel: jms/updateOrdineChannel data[
        {
            foo:"bar"
        }
    ]
    2017-11-07 17:25:56,585 DEBUG [ ] JMSMessageDispatcher Decoded jmsMessage for channel: jms/updateOrdineChannel data[
        {
            foo:"bar"
        }
    ]
    Do these mean that there are still some duplicated messages?
    Last edited by claudiobosticco; 7 Nov 2017, 08:46.

    Leave a comment:


  • Isomorphic
    replied
    So, that's definitely possible with some brokers in certain configurations, but there was definitely a bug with Remote debugging specifically in clustered configs. Last night's (Nov 5) build addresses this. If you have some time to try it out, that would be great. That build also contains additional JMS improvements to help with server load with a high number of clients.

    Either way - glad you have a working solution - thanks for providing all this context to help us troubleshoot!

    Leave a comment:


  • claudiobosticco
    replied
    Hello, actually there was a typo in my config, instead the realtime messaging works in cluster with this configuration for the channel:

    Code:
      
    <Resource
                  name="jms/myChannel"
                  type="javax.jms.Topic"
                  factory="com.rabbitmq.jms.admin.RMQObjectFactory"
                  destinationName="myChannel"/>
    Last edited by claudiobosticco; 6 Nov 2017, 02:13.

    Leave a comment:


  • Isomorphic
    replied
    Just to be clear, the routing issue is just for the remote debugging logic which uses temporary topics. JMS delivery to durable queues and topics is fine (which is why your use of Messaging in general works, but the Developer Console remote debugging is broken).

    Leave a comment:


  • Isomorphic
    replied
    Yeah, We are reproducing this issue here as well. There is in fact a problem with the way the messages our routed. There will be a fix to this issue as well as the duplicate message deliver to the browser (which you mentioned earlier). The dups are weeded out by some defensive logic on the client, but it's inefficient. There are a few other inefficiencies that are being addressed as part of this work as well that should substantially speed up JMS configurations with many clients.

    Will update this once we have a definitive build to point you to, but work is almost complete. Expect it in the next couple of days.

    Leave a comment:


  • claudiobosticco
    replied
    Hello, yes, both my tomcat instances are started with that option (see the text in bold):
    Code:
    tomcat@srvubuntu8:~$ ps aux | grep java
    
    tomcat   22744 37.2 44.9 6206144 1819608 ?     Sl   17:06   8:37 /usr/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Xms512M -Xmx3072M -server -XX:+UseParallelGC -XX:PermSize=512m -XX:MaxPermSize=512m -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n [B]-Dorg.apache.activemq.SERIALIZABLE_PACKAGES=*[/B] -classpath /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start
    But thanks for the heads up about activeMQ. Actually I don't want to set up a new environment with a jms broker which has no future.

    So, I've installed RabbitMQ (and disabled activeMQ). After putting these jars in the tomcat /lib:
    • amp-client-4.3.0.jar
    • rabbitmq-jms-1.7.0.jar
    • geronimo-jms_1.1_spec-1.1.1.jar
    • slf4j-api-1.6.1.jar
    and configuring the ConnectionFactory like this:
    Code:
    <Resource name="jms/ConnectionFactory"
                         type="javax.jms.ConnectionFactory"
                         factory="com.rabbitmq.jms.admin.RMQObjectFactory"
                        username="foo"
                        password="bar"
                        virtualHost="/"
                        host="myBrokerHost"/>
    and my channel:
    Code:
        <Resource  name="jms/updateOrdineChannel"
                       type="javax.jms.Queue"
                      factory="com.rabbitmq.jms.admin.RMQObjectFactory"
                      destinationName="updateOrdineChannel"
                      amqp="true"
                      amqpQueueName="updateOrdineChannel"/>
    I've got a situation where the messaging and the remote debug seem to work but only when the browsers are served by the same tomcat instance.
    Also, note that the messaging on the updateOrdineChannel works even without the above channel definition. Is this expected?

    I'm still using v11.1p_2017-10-18 build. Have you already released a build with better debugging? I hope will help me to configure everything correctly.

    EDIT: see post #26 for working channel config.
    Last edited by claudiobosticco; 6 Nov 2017, 02:12.

    Leave a comment:


  • Isomorphic
    replied
    On further testing, I take it back - the special serialization flag only needs to be enabled on the Tomcat JVM. Can you please double check that that option is definitely enabled in your JVM invocation?

    On a related note, we will release an update to at least 11.1 in the next few days to enable better debugging and possibly include a workaround to requiring this flag for ActiveMQ - depending on feasibility and testing outcomes.

    Leave a comment:


  • Isomorphic
    replied
    Ok, I'm quite sure now that the serlialization "security" feature is the actual problem. In order to fix the problem, you not only need to start Tomcat, but also the broker with this option:

    Code:
    -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=*
    I'm going to see if we can remove this constraint by eliminating the use of the non-jvm-native org.apache.commons.collections4.map.LinkedMap, but this will require some testing to make sure we don't break anything else. We will also improve the JMS logging and error handling a bit to make it easier to track this sort of thing down in the future.

    As an aside, the RabbitMQ works fine out of the box. And ActiveMQ may not have a future: http://activemq.apache.org/new-features-in-60.html.

    Leave a comment:

Working...
X