9 Replies Latest reply on Jun 12, 2012 11:44 AM by ataylor

    Randomly entity not found 'onMessage'

    marhat

      Hello,

       

      I have a question concerning HornetQ running in JBoss 6.0.0.

       

      I have a scenario whereby I want to track a series of messages posted to hornetq in an "ImportLog" entity I have created. To give a little further detail, the messages posted to HornetQ contain records that are to be imported into the database. For performance reasons, we like to limit the batch of records inserted at any one time. If our limit is 500 and a user decides to import 2000, we would create 4 messages for example.

       

      The ImportLog is a simple entity that helps us track and record stats of the import as a whole. So each message that processes will load the ImportLog entity and potentially update. The import queue has the maxSessions property set to 1, which means that the messages correctly process one-at-a-time.

       

      Before any messages are posted to HornetQ, the ImportLog entity is created. The id of the ImportLog entity is then passed as part of the object message.

       

      Radomly, everything will work perfectly fine. However, very frustratingly, the first message picked up will often claim that it cannot find the ImportLog entity (i.e. ImportLog log = em.find(ImportLog, id)). It appears that the message is being consumed before Hibernate has committed the import log to the database? Interestingly subsequent messages will find the ImportLog perfectly.

       

      I have tried various things to "force" hibernate to commit the entity so that the first message will always find the entity in the database. This includes calling flush(), manually committing the transaction after the ImportLog entity is initially created, along with other tricks I have read about that include calling session.evict() and session.update().

       

      Has anybody else found this problem? Is there a way to overcome or am I missing something / abusing functionality here?

       

      Any thoughts would be greatly appreciated

        • 1. Re: Randomly entity not found 'onMessage'
          ataylor

          I dont really see this as a hornetq question, I would ask on the Hibernate forums

          • 2. Re: Randomly entity not found 'onMessage'
            leosbitto
            • 3. Re: Randomly entity not found 'onMessage'
              clebert.suconic

              @Leos I think on this case it was just some misuse on Hibernate. Or even maybe not rolling back properly the message on the Transaction Manager. I believe this is well established how it should behave at this point.

              • 4. Re: Randomly entity not found 'onMessage'
                marhat

                @Clebert, i disagree that this case is "just some misuse on Hibernate".

                 

                The link provided by Leos descibes exactly the problem I am facing. In the short term, I am having to rewrite code and lose minor functionality to overcome. In the long term, I am investigating the solutions put forward in the link.

                • 5. Re: Randomly entity not found 'onMessage'
                  clebert.suconic

                  We just set the transaction at the onMessage, all you need to do is to configure hibernate properly to the transaction manager. What makes it a configuration issue outside of HornetQ.

                  • 6. Re: Randomly entity not found 'onMessage'
                    leosbitto

                    As I have described in that Jira issue JMS_SPEC-28, if everybody (HornetQ and Hibernate in this case) just states "not my issue", it is not going to work. In this case it would help if HornetQ would be able to implement the marker interface com.arjuna.ats.jta.resources.EndXAResource for its XAResource. I understand that creating such direct dependency on Arjuna's API is not acceptable, but providing some extensibility (API for providing an optional wrapper for XAResource maybe?) could be possible, couldn't it?

                    • 7. Re: Randomly entity not found 'onMessage'
                      jbertram

                      To be fair, this really isn't a HornetQ issue.  Even Nigel said, "...this doesn't sound like something that can be tackled at the level of an individual resource manager such as JMS."  The problem should be addressed by the JTA specification.

                       

                      As you pointed out, JBossTS (the transaction manager used by JBoss AS) has an extension that will allow more control of this, but there are handful of problems with having our XAResource implementation also implement com.arjuna.ats.jta.resources.EndXAResource.  These are the issues I can think of off the top of my head:

                      • We'd need to provide 3 implementations - one that implemented com.arjuna.ats.jta.resources.EndXAResource, one that implemented com.arjuna.ats.jta.resources.StartXAResource, and one that didn't implement either (which would be default).  We wouldn't want to change the default behavior or only provide the option to make it the last resource committed.
                      • This would only help when JBossTS was used as the TM.  Since HornetQ can be deployed embedded in other containers and accessed remotely from other containers even when run within JBoss AS (or standalone) this is far from a complete solution.
                      • This is essentially a hack which likely brings unwanted dependencies.

                       

                      As I understand it, this is a known weakness in JTA (which is why the JBossTS work-around exists).  However, I'm not sure what measures have been undertaken to resolve it - if any.  I'm interested to see what happens on http://java.net/jira/browse/JAVAEE_SPEC-1.

                       

                      I'll ask Clebert what he wants to do here as far as it concerns HornetQ as it's not clear to me if this is really something we want to do.

                      • 8. Re: Randomly entity not found 'onMessage'
                        ataylor

                        also read http://jbossts.blogspot.co.uk/2011/04/messagingdatabase-race-conditions.html which has some good insight into it. It also suggest a way around the issue altho slightly clunky.

                         

                        As far as I am aware this is being tackled in the JTA 1.2 spec.

                         

                        maybe one thing you can do is use a user managed tx and enlist the resources in the order you want them commited.

                        • 9. Re: Randomly entity not found 'onMessage'
                          ataylor

                          also according to the TS team EndXAResource is an internal API that has some side effects that we wouldnt want